Cooled Electronic System

ABSTRACT

A sealable module, cooled electronic system and method are described relating to cooling a heat generating electronic device. The sealable module is adapted to be filled with a first cooling liquid and a heat transfer device having a conduction surface defines a channel for receiving a second cooling liquid. In one embodiment, at least a portion of the conduction surface or housing is shaped in conformity with the shape of the electronic component. Control of the second cooling liquid is also described. Transferring heat between the second cooling liquid and a third cooling liquid features in embodiments. A method of filling a container with a cooling liquid is further detailed.

TECHNICAL FIELD OF THE INVENTION

This invention relates to a module or capsule for housing an electroniccomponent that generates heat in operation, a method of cooling such anelectronic component, a method of cooling an electronic device, a cooledelectronic system, and a method of filling a container for an electronicdevice with a cooling liquid. The invention is particularly applicable,for example, to computer processors and motherboards.

BACKGROUND TO THE INVENTION

Electronic components and in particular computer processors generateheat in operation, which can lead to overheating and consequent damageto the component and other parts of the system. It is thereforedesirable to cool the component to transfer the heat away from thecomponent and maintain the component temperature below the maximumoperating temperature that is specified for correct and reliableoperation of the component.

This issue especially concerns data processing or computer servercentres, where a substantial number of computer processors areco-located and intended for reliable, continuous operation over a longtime period. These centres may typically contain many server units,occupying multiple equipment racks and filling one or more rooms. Eachserver unit contains one or more server board. A single server board candissipate many hundreds of watts of electrical power as heat. Inexisting systems, the energy required to transfer heat continuously soas to maintain correct operation can be of the same order of magnitudeas the energy required to operate the servers.

The heat generated can be transferred to a final heat sink external tothe building in which the processors are located, for example to theatmospheric air surrounding the building. Current implementationstypically rely on air as the transfer medium at one or more stagesbetween the processors and the final heat sink.

However, it is difficult to use air as a transfer medium for such alarge quantity of heat, without imposing significant limitations on thebuilding infrastructure. This is because the rate at which heat can betransferred increases with: increasing temperature difference (ΔT)between the heat source (such as the server boards, in particular thecomputer processors) and final heat sink; and decreasing thermalresistance of the path or paths thermally connecting the heat source andfinal heat sink.

Some known technologies for dealing with this difficulty are designed tocontrol the environmental conditions of the location at which theprocessors are housed. Air handling techniques are currently often used,for example: vapour-compression refrigeration (“air conditioning”) ofthe air that reduces the local air temperature to increase the localtemperature difference; and air pressurization (by the use of fans) toincrease the air flow rate and thereby reduce the thermal resistance.Further heat exchange stages may be used to transfer heat extracted fromthe local air to a final heat sink, such as atmospheric air.

However, these approaches can be inefficient, as the use of airconditioning can require substantial amounts of electrical power tooperate. These approaches can also make the location unpleasant forpeople, due to the local temperature and noise.

Furthermore, air flow rates and air temperatures may have to be limited,for example maintaining temperature above the “dew point” to preventwater vapour condensing out of air that may damage sensitive electroniccomponents. For these reasons, servers are currently commonlydistributed sparsely in order to reduce the heat density and improvelocal air flow, thereby reducing the thermal resistance.

Cooling the electronic components using a liquid that is brought intocontact with the electronic components can be used to increase serverdensity, reduce cooling costs or both.

An existing technique for cooling electronic components using a liquidis described in US-2007/109742 and GB-A-2432460. A computer processorboard is housed inside an airtight container. A coolant liquid,preferably oil, is pumped through the container. The processor board islocated at the bottom of the container and an evaporator coil ispositioned at the top of the container, such that convection currentsare produced in the coolant liquid. The coolant liquid is heated by theprocessor board and resultant vapour flows into a condenser. Thecontainer is positioned such that the circuit board inside lies in ahorizontal plane to allow convection of heat from the components.

Using a condenser to provide refrigeration increases the complexity andcost of the system, and introduces further limitations on the systemimplementation.

WO-2006/133429 and US-2007/034360 describe an alternative known approachfor cooling electronic components. The electronic component is sealedinside a container filled with a liquid and a thermally conductive plateis provided as part of the container in contact with the liquid. Thethermally conductive plate conducts heat from the liquid to the outsideof the container. Although this is designed for independent operation,the thermally conductive plate can be coupled to a further heatexchanger for additional cooling of the electronic component.

This alternative arrangement reduces the complexity of the system incomparison with approaches requiring pumped fluids inside the container.However, this does not significantly address the difficulty in reducingthe thermal resistance between the heat source and the final heat sink.Even if the temperature difference is increased, the total thermalresistance will still be significant.

SUMMARY OF THE INVENTION

Against this background and in a first aspect, there is provided asealable module for containing one or more heat generating electroniccomponents. The module comprises: a housing; a heat transfer devicehaving a conduction surface, the housing and the conduction surfacetogether defining a volume in which a first cooling liquid can belocated, the heat transfer device further defining a channel forreceiving a second cooling liquid, the conduction surface separating thevolume and the channel to allow conduction of heat between the volumeand the channel through the conduction surface; and an electroniccomponent located in the volume. At least a portion of the conductionsurface or housing is shaped in conformity with the shape of theelectronic component.

By conforming the shape of the housing, the conduction surface or bothto the shape of the electronic component, the efficiency of heattransfer between the first cooling liquid and second cooling liquid issignificantly increased. This makes it possible to maintain the firstcooling liquid in a liquid state up to a high level of heat output fromthe electronic component.

In the preferred embodiment, the sealable module further comprises atleast one electronic component which generates heat when in use, and afirst cooling liquid, located in the volume.

The second liquid coolant is caused to flow in the channel in directcontact with the conduction surface. The second liquid coolant ispreferably pumped. Transferring heat by conduction from a first coolingliquid in the volume to the second cooling liquid in the channel reducesthe thermal resistance significantly. This increases the efficiency ofheat transfer, making such a system scalable and applicable to systemswhich generate large quantities of heat, such as data processingcentres. Moreover, the reduced thermal resistance of this system gainedby using a conduction surface to transfer heat allows the coolant to bemaintained in a liquid state at all times, thereby avoiding the need forvapour-cycle refrigeration that increases the complexity and cost of thesystem.

Also, the electricity consumption for cooling is reduced by mitigatingor even eliminating the need for vapour-compression refrigeration. Thisalso allows the density of electronic components and electronic circuitboards, such as server boards, to be increased.

For a given density of servers and components, a cooling systemdesirably removes sufficient heat from each component to keep it withinits intended operating temperature range, but no more than that. Devicesthat generate less heat need less cooling than those that generatelarger amounts. Cooling below a level necessary for satisfactoryoperation will normally consume unnecessary additional energy and istherefore less than optimally efficient.

Beneficially, the channel has an area of contact with the conductionsurface defining a channel width, and wherein the minimum channel widthis significant in comparison with the dimensions of the conductionsurface.

Additionally or alternatively, the channel interfaces with theconduction surface along a path having at least one change in direction.Preferably, the path has a straight portion, and wherein the minimumchannel width is at least 10% of the length of the straight portion ofthe path. In other embodiments, the minimum channel width may be atleast: 10%; 20%; 30%; 40%; or 50% of the length of the straight portionof the path. Additionally or alternatively, the path comprises a mainpath and a branch path, the branch path being connected to the main pathat at least one point. More turbulent flow of liquid can improve heattransfer.

In a second aspect, there is provided a method of cooling an electroniccomponent, comprising: providing a module comprising a housing and aheat transfer device having a conduction surface, the housing and theconduction surface together defining a volume; housing the electroniccomponent within the volume; filling the volume with a first coolingliquid; and conducting heat between the first cooling liquid and asecond cooling liquid through the conduction surface, the first coolingliquid and second cooling liquid being located on either side of theconduction surface. At least a portion of the conduction surface orhousing is shaped in conformity with the shape of the electroniccomponent.

Preferably, the step of conducting heat from the first cooling liquid toa second cooling liquid is configured such that the first cooling liquidand second cooling liquid remain in liquid state.

In a third aspect, there may be provided sealable module kit,comprising: a housing; a heat transfer device having a conductionsurface, the heat transfer device being configured to couple to thehousing such that the conduction surface and housing define a volume inwhich a first cooling liquid can be located, the heat transfer devicefurther defining a channel for receiving a second cooling liquid suchthat when the heat transfer device is coupled to the housing, theconduction surface separates the volume and the channel to allowconduction of heat between the volume and the channel through theconduction surface; and an electronic component for location in thevolume. At least a portion of the conduction surface or housing isshaped in conformity with the shape of the electronic component.

A number of additional features can be applied as specified by any oneof the first, second or third aspects specified above. These will bedescribed below.

Beneficially, the conduction surface has at least one projection intothe first volume for conducting heat between the volume and the channel.The use of at least one projection increases the surface area of theconduction surface and can allow closer conformity between theconduction surface and the shape of the electronic component. Theseimprove the efficiency of heat conduction.

Where the sealable module includes the electronic component, theconduction surface preferably has at least one projection into the firstvolume for conducting heat between the volume and the channel, the atleast one projection being arranged in conformity with the shape of theelectronic component. This further improves heat conduction efficiency,by: reducing the space between the component and the conduction surface;increasing total projection surface area to reduce thermal resistance inthe heat flow path; reducing the volume of cooling liquid required forefficient cooling; permitting the increased use of materials with poorerconductivity but reduced cost or weight or both (e.g. plastic). Inparticular, efficiency of cooling is improved if the cooling liquidcomes quickly into contact with the conduction surface.

Optionally, the conduction surface is made from a synthetic plasticmaterial, which is desirably thermally conductive. Additionally oralternatively, the housing may be made from a synthetic plasticmaterial. Preferably, this is thermally insulating. In embodiments, theheat transfer device may be made from a synthetic plastic material.

In some embodiments, the sealable module further comprises a componentheat sink coupled to the electronic component, having at least oneprojection arranged to cooperate with the at least one projection of theconduction surface.

Advantageously, the at least one projection of the conduction surfacecomprises a fin arrangement. Alternatively or additionally, the at leastone projection of the conduction surface comprises a pin arrangement. Inthe preferred embodiment, the at least one projection comprises apin-fin arrangement. The pin-fin projections may vary in height.Preferably, they are fin-shaped with a rectangular cross section. Morepreferably, the projections do not cover the whole conduction surface.

Preferably, a flow diverter is located in the volume. This may take theform of a baffle plate or other passive flow control mechanism. Such afeature has the purpose of deflecting hot rising plumes of liquid awayfrom components directly above that might otherwise over-heat.

Optionally, a plurality of electrical conductors are located in thevolume, the plurality of electrical conductors being arranged to detectthe level of the first cooling liquid. These electrical conductorspreferably take the form of a pair of rods at the top of the moduleextending down into the volume. These can act as a capacitor, whosevalue changes as the liquid level alters (by application of thedielectric effect). By attaching a suitable electronic circuit, theliquid level for the first cooling liquid can be deduced. This allowsthe operator to know when the level is below normal for the currentliquid temperature and indicates the likelihood of a leak. The samedevice could be used to determine liquid level during initial filling(or topping up) of the module.

Additionally or alternatively, a transparent window in the housing isused to observe the level directly. A further optional improvement wouldbe to build the sensor and associated circuit into a circuit board onwhich the electronic component is mounted or adjacent to the electroniccomponent.

In some embodiments where the conduction surface defines the channel,the shape of the channel is arranged in conformity with the shape of theelectronic component. This allows further improvements in efficiency ofheat transfer between the first cooling liquid and second coolingliquid.

In the preferred embodiment, the heat transfer device is formed in onepiece. A one piece assembly for the heat transfer device may not requiremaintenance and could be constructed from a cold plate part into whichchannels were pre-formed and a sealing plate welded or otherwise adheredonto the cold plate part. This eliminates screws and gaskets and reducesthe likelihood of liquid leakage.

In some embodiments, the heat transfer device further comprises a basepart coupled to the conduction surface and defining the channel forreceiving the second cooling liquid.

In one embodiment, the housing and conduction surface defines an innerchamber, and the heat transfer device further comprises an outerchamber, defining the channel.

The use of an outer chamber that defines the channel for receiving asecond cooling liquid and which is arranged to cooperate with the innerhousing to provide the conduction surface advantageously provides acompact module. Such an arrangement may additionally allow the width ofthe channel to be defined or adjusted to meet the heat transferrequirements. Optionally, the outer chamber is made from a syntheticplastic material.

Advantageously, the sealable module further comprises an insulationlayer covering at least part of the housing. Preferably, the insulationlayer is within the volume. Additionally or alternatively, the sealablemodule may comprise an insulation layer covering at least part of thehousing, exterior to the volume. The exterior insulation may compriseflexible foam. This has the further advantage of being able to supportthe connectors for liquid input, output or both, whilst allowing anelement of flexing of the pipes. This would make insertion into anassociated rack easier by allowing more tolerance in the sliding parts.

Preferably, more than one electronic component may be located in thevolume. In the preferred embodiment, the sealable module comprises aplurality of electronic components, at least one of which generates heatwhen in use, and further comprising a circuit board holding theplurality of electronic components.

By immersing the circuit board in a carefully selected liquid, it isisolated from damage by airborne pollutants or water that mightotherwise condense out of the atmosphere, or leak elsewhere. Pollutantspresent in air or dissolved in water can readily attack the fine wiringon circuit boards, for example. Also, other heat generating componentssuch as power supplies, DC-DC converters and disk drives can beencapsulated and cooled. Where the at least one electronic componentincludes a disk drive, the disk drive is preferably a solid statedevice. Devices with moving parts are undesirable for immersion in aliquid.

The first cooling liquid preferably occupies a portion of the volume ofthe sealable module, such that there is volume available for expansionof the liquid upon heating without significantly increasing the pressurein the volume.

Preferably, the sealable module further comprises a protective membranepositioned between the circuit board and the housing, the protectivemembrane being arranged to prevent liquid flow between the housing andthe circuit board. This increases the thermal resistance through thisundesirable path and reduces the volume of the coolant required to fillthe volume, whilst allowing for the presence of small components on therear of a circuit board. In the preferred embodiment, the protectivemembrane is deformable.

Advantageously, at least a part of the circuit board is integrallyformed with the housing. The electronic circuit board could beintegrated with part of the module housing, for example just using sidewalls. The interconnections could then pass directly through the circuitboard to appear on the outer face of the housing, eliminating the needfor cables to be sealed at the point of exit with the module.Optionally, the circuit board and walls are constructed as a single itemthus reducing further the sealing issue, for example using a fibreglassmoulding.

In the preferred embodiment, the sealable module further comprises: afilling inlet to the module, located in the housing and through which aliquid can be received into the volume; and a seal to the filling inlet.This allows quick filling or re-filling of the sealable module volumewith cooling liquid in field.

Preferably, the sealable module comprises a pressure relief valve,located in the housing and arranged to allow liquid flow out from thevolume when the pressure within the volume exceeds a predefined limit.

Optionally, the channel has an area of contact with the conductionsurface defining a channel width, and wherein the minimum channel widthis significant in comparison with the dimensions of the conductionsurface.

Additionally or alternatively, the channel interfaces with the at leasta portion of the conduction surface along a path having at least onechange in direction. Preferably, the path has a straight portion, andwherein the minimum channel width is at least 10% of the length of thestraight portion of the path. In other embodiments, the minimum channelwidth may be at least: 10%; 20%; 30%; 40%; or 50% of the length of thestraight portion of the path. Additionally or alternatively, the pathcomprises a main path and a branch path, the branch path being connectedto the main path at at least one point.

There may also be provided a cooled electronic system, comprising: thesealable module as described herein; an electronic component located inthe volume; a first cooling liquid located in the volume; a heat sink; apumping arrangement, arranged to allow a second cooling liquid to flowthrough at least a portion of the channel of the sealable module to theheat sink at a predetermined flow rate; a temperature sensor, arrangedto determine a temperature of the electronic component; and a controllerarranged to control at least one of: the pumping arrangement; and theportion of the channel through which the second cooling liquid flows,such that the temperature of the electronic component is controlled soas not to exceed a predetermined maximum operating temperature.

Beneficially, the conduction surface has at least one projection intothe first volume for conducting heat between the volume and the channel.Where the sealable module includes the electronic component, theconduction surface preferably has at least one projection into the firstvolume for conducting heat between the volume and the channel, the atleast one projection being arranged in conformity with the shape of theelectronic component. In some embodiments, the sealable module furthercomprises a component heat sink coupled to the electronic component,having at least one projection arranged to cooperate with the at leastone projection of the conduction surface. Advantageously, the at leastone projection of the conduction surface comprises a fin arrangement.Alternatively or additionally, the at least one projection of theconduction surface comprises a pin arrangement. In the preferredembodiment, the at least one projection comprises a pin-fin arrangement.

A fourth aspect may be found in a method of cooling an electronicdevice, comprising: providing a module comprising a housing and a firstheat transfer device having a conduction surface, the housing and theconduction surface together defining a volume, the volume being filledwith a first cooling liquid and having the electronic device locatedtherein; operating the electronic device within the volume; transferringheat generated by the electronic device from the first cooling liquid toa second cooling liquid through at least a portion of the conductionsurface; transferring heat from the second cooling liquid to a heat sinkusing a second heat transfer device; and setting one or both of: theflow rate of the second cooling liquid from the conduction surface tothe heat sink; and the portion of the conduction surface through whichheat is transferred to the second cooling liquid, such that thetemperature of the electronic device is controlled so as not to exceed apredetermined maximum operating temperature.

By setting the flow rate, area of heat transfer or both, the thermalresistance can advantageously be increased or reduced as needed toprovide the desired temperature difference. This allows the electroniccomponent to be maintained at a temperature no greater than its maximumoperating temperature, even if the temperature difference decreases, forexample, if the final heat sink temperature increases (such as when anatmospheric final heat sink is used). Optionally, the conduction surfaceis made from a synthetic plastic material.

Optionally, the method further comprises: determining a temperature ofthe electronic component. The step of setting is carried out on thebasis of the determined temperature. This is a form of dynamicadjustment based on measured temperature.

Additionally or alternatively, the step of setting comprises at leastone of: the flow rate of the second cooling liquid from the conductionsurface to the heat sink; and the portion of the conduction surfacethrough which heat is transferred to the second cooling liquid being setto a predetermined level on the basis of a predicted predeterminedmaximum operating temperature for the electronic device. In suchembodiments, the flow rate, portion of the conduction surface or bothare pre-set based on a predicted heat output or heat output range fromthe electronic device.

In a fifth aspect, there is provided a cooled electronic system,comprising: a sealed container comprising: a housing; an electronicdevice; and a first cooling liquid; a first heat transfer devicedefining a first channel for receiving a second cooling liquid, thefirst heat transfer device being configured to transfer heat between thefirst cooling liquid and the first channel through at least a portion ofa conduction surface; and a piping arrangement, configured to transferthe second cooling liquid to and from the first heat transfer device.The system is configured to set one or both of: the flow rate of thesecond cooling liquid through the first channel; and the portion of theconduction surface through which heat is transferred to the secondcooling liquid, such that the temperature of the electronic device iscontrolled so as not to exceed a predetermined maximum operatingtemperature. Advantageously, the second cooling liquid flows from theconduction surface to a heat sink.

A sixth aspect may be provided by a method of cooling an electronicdevice, comprising: operating the electronic device within a container,the container also comprising a first cooling liquid, such that heatgenerated by the electronic device is transferred to the first coolingliquid, the container being sealed to prevent leakage of the firstcooling liquid; transferring heat between the first cooling liquid and asecond cooling liquid in a first heat transfer device; piping the secondcooling liquid from the first heat transfer device to a second heattransfer device; transferring heat between the second cooling liquid anda third cooling liquid in the second heat transfer device; and pipingthe third cooling liquid to a heat sink.

Thus, three stages of liquid cooling are provided, which allows the flowrate and pressure of the second cooling liquid and third cooling liquidto be independently controlled. The pressure of second cooling liquidcan therefore be reduced to further mitigate the risk of leakage of thisliquid. Since these liquids are in close proximity to the electroniccomponents, leakage is undesirable. Also, the flow rates canadvantageously be controlled based upon the level of heat generated toimprove the efficiency of heat transfer at each stage.

Preferably, the step of transferring heat between the first coolingliquid and the second cooling liquid is carried out by conduction.

In the preferred embodiment, the method further comprises: controllingthe flow rate of the second cooling liquid, such that the temperature ofthe electronic device does not exceed a predetermined maximum operatingtemperature. Additionally or alternatively, the method may alsocomprise: controlling the flow rate of the third cooling liquid, suchthat the temperature of the electronic device does not exceed apredetermined maximum operating temperature. This allows the flow rateof second cooling liquid to be matched to the level or rate of heatgeneration.

Where the first heat transfer device comprises a conduction surface, themethod optionally further comprises setting the portion of theconduction surface through which heat is transferred to the secondcooling liquid, such that the temperature of the electronic device iscontrolled so as not to exceed a predetermined maximum operatingtemperature. This can be achieved, for example, using multiple channelsin the conduction surface for carrying the second cooling liquid andappropriate control valves or baffle plates to determine the channel orchannels along which the second cooling liquid should flow, or tobalance the flow rate of cooling liquid appropriately between differentchannels in order to maintain the temperature of the electronic devicebelow a threshold. The channels thereby provide similar or differentliquid flow rates over different areas and thus different heat transferrates from different parts of the conduction surface.

In some embodiments, the method may further comprise controlling atleast one of: the flow rate of the second cooling liquid; and the flowrate of the third cooling liquid, such that the temperature of theelectronic device does not exceed a predetermined maximum operatingtemperature and such that, during a first time period, the heat transferrate between the second cooling liquid and the third cooling liquid orbetween the third cooling liquid and the heat sink does not go above apredetermined maximum rate, and such that during a second, later timeperiod, the heat transfer rate between the second cooling liquid and thethird cooling liquid or between the third cooling liquid and the heatsink may go above the predetermined maximum rate.

Optionally, the step of controlling is carried out such that thetemperature of the electronic device does not exceed a predeterminedmaximum operating temperature and such that, during a first time period,the temperature of at least one of: the second cooling liquid; and thethird cooling liquid does not go below a predetermined minimum averagetemperature, and such that during a second, later time period, thetemperature of the second cooling liquid and the third cooling liquidmay go below the predetermined minimum average temperature.

In a further aspect, there is provided a method of cooling an electronicdevice, comprising: operating the electronic device within a container,the container also comprising a first cooling liquid, such that heatgenerated by the electronic device is transferred to the first coolingliquid, the container being sealed to prevent leakage of the firstcooling liquid; transferring heat between the first cooling liquid and asecond cooling liquid in a first heat transfer device; transferring heatbetween the second cooling liquid and a heat sink; and controlling theheat transfer rate between the second cooling liquid and the heat sink,such that the temperature of the electronic device does not exceed apredetermined maximum operating temperature and such that, during afirst time period, the heat transfer rate does not go above apredetermined maximum rate, and such that during a second, later timeperiod, the heat transfer rate may go above the predetermined maximumrate.

An advantageous feature of the system used in the method is the highthermal capacity of the cooling arrangement. This has a number ofbenefits and opportunities. Failure of a part of the system will notlead to immediate component damage. The temperature will rise but notquickly, giving maintenance staff more time to isolate the faulty partand minimise further failures. Similarly, the system is able to copewith environments with high diurnal ambient temperature variations. Heatbuilt up during the day can be tolerated by the system without exceedingmaximum component operating temperatures. The heat can be safelydissipated at night to the cold. The system flow management algorithmmay be arranged to take account of such high diurnal ambient variations.

Advantageously, a method of cooling an electronic system is provided,comprising: carrying out the method steps of cooling an electronicdevice in accordance with this sixth aspect; operating a secondelectronic device within a second container, the second container alsocomprising a fourth cooling liquid, such that heat generated by theelectronic device is transferred to the fourth cooling liquid, thesecond container being sealed to prevent leakage of the fourth coolingliquid; and transferring heat between the fourth cooling liquid and afifth cooling liquid in a third heat transfer device. Optionally, themethod further comprises controlling the flow rate of the fifth coolingliquid.

A unit with two containers may be heavy and difficult to install in arack, as they could exceed health and safety limits for a one-personlift. In one embodiment, first and second containers are used and thesecontainers are provided back-to-back. In other words, the connectors ofthe first container are positioned adjacent to the connectors of thesecond container. Back-to-back containers with centralized cabling andliquid pipes in a rack would allow units with one module and singleperson lift.

Preferably, the second cooling liquid and the fifth cooling liquid arecombined. This allows efficient cooling of multiple electronic devicesusing separate first cooling stages, but a common second stage ofcooling. More preferably, the method further comprises piping the secondcooling liquid from the first heat transfer device and the fifth coolingliquid from the third heat transfer device to a plenum chamber. Mostpreferably, the second cooling liquid and the fifth cooling liquid arecombined before arrival at the plenum chamber. Optionally, the methodalso comprises: piping the second cooling liquid from a plenum chamberto the first container and piping the fifth cooling liquid from theplenum chamber to the second container.

Optionally, the method further comprises controlling the flow rate ofthe combined second cooling liquid and fifth cooling liquid, such thatthe temperature of the first electronic device does not exceed a firstpredetermined maximum operating temperature and such that thetemperature of the second electronic device does not exceed a secondpredetermined maximum operating temperature.

In this embodiment, individual flow control, per module, is notnecessary. The overall flow rate of the combined liquid can becontrolled with satisfactory results, when combined with pre-set flowbalancing of liquid to each cooling unit, for example, by means ofbaffles in the supply side plenum chamber.

In an alternative embodiment, the method further comprises: piping thefifth cooling liquid from the third heat transfer device to a fourthheat transfer device; and transferring heat between the fifth coolingliquid and the third cooling liquid in the fourth heat transfer device.

In an seventh aspect, there may be found in a cooled electronic system,comprising: a sealed container comprising: a housing; an electronicdevice; and a first cooling liquid; a first heat transfer devicedefining a first channel for receiving a second cooling liquid, thefirst heat transfer device being configured to transfer heat between thefirst cooling liquid and the first channel; and a second heat transferdevice comprising a second channel for receiving the second coolingliquid from the first channel, and a third channel for receiving a thirdcooling liquid for coupling to a heat sink, the second heat transferdevice being configured to transfer heat between the second channel andthe third channel.

Preferably, the first heat transfer device comprises a conductionsurface, the housing and the conduction surface together defining avolume in which the electronic component and the first cooling liquidare located. More preferably, the conduction surface separates thevolume and the first channel to allow conduction of heat between thevolume and the channel through the conduction surface.

Beneficially, at least a portion of the conduction surface or housing isshaped in conformity with the shape of the electronic device.

Advantageously, the conduction surface has at least one projection forreceiving heat from the first cooling liquid. In the preferredembodiment, the at least one projection is arranged in conformity withthe shape of the electronic device. Optionally, the cooled electronicsystem further comprises a component heat sink coupled to the electronicdevice and comprising at least one projection arranged to cooperate withthe at least one projection of the conduction surface.

Beneficially, the at least one projection of the conduction surfacecomprises a fin arrangement. Alternatively or additionally, the at leastone projection of the conduction surface comprises a pin arrangement. Inthe preferred embodiment, the at least one projection comprises apin-fin arrangement.

In one embodiment, the heat transfer device further comprises a basepart coupled to the conduction surface and defining the channel forreceiving the second cooling liquid.

Advantageously, the conduction surface is made from a synthetic plasticmaterial. Additionally or alternatively, the housing may be made from asynthetic plastic material. In embodiments, the base part of the heattransfer device may be made from a synthetic plastic material.

Optionally, the module further comprises: a filling inlet to thecontainer, through which the first cooling liquid can be received; and aseal to the filling inlet. Additionally or alternatively, the modulefurther comprises a pressure relief valve, arranged to allow outflow ofthe first cooling liquid from the container when the pressure within thecontainer exceeds a predefined limit.

Where the electronic device has an elongate axis, the conduction surfacepreferably has an elongate axis arranged in conformity with the elongateaxis of the electronic device to allow conduction of heat between thevolume and the channel through the conduction surface.

Optionally, the cooled electronic system further comprises a flowcontrol device, arranged to control the flow rate of the second coolingliquid, such that the temperature of the electronic device does notexceed a predetermined maximum operating temperature.

Additionally or alternatively, the cooled electronic system may furthercomprise a flow control device, arranged to control the flow rate of thethird cooling liquid, such that the temperature of the electronic devicedoes not exceed a predetermined maximum operating temperature.

In some embodiments, the sealed container is a first sealed container,and the cooled electronic system further comprises: a second sealedcontainer comprising: a second housing; a second electronic device; afourth cooling liquid for receiving heat from the second electronicdevice; and a third heat transfer device comprising a fourth channel forreceiving a fifth cooling liquid, the third heat transfer device beingconfigured to transfer heat from the fourth cooling liquid to the fourthchannel.

Preferably, the first channel and the fourth channel are coupled tocombine the second cooling liquid and the fifth cooling liquid.Optionally, the cooled electronic system further comprises a plenumchamber, arranged to collect the combined second cooling liquid andfifth cooling liquid.

Advantageously, the cooled electronic system further comprises a flowcontrol device, arranged to control the flow rate of the combined secondcooling liquid and fifth cooling liquid, such that the temperatures ofthe first electronic device and the second electronic device do notexceed first and second predetermined maximum operating temperaturesrespectively.

Preferably, the flow control device comprises a flow divertingarrangement, the flow diverting arrangement being configured to set theflow rate of the second cooling liquid such that the temperature of thefirst electronic device does not exceed a first predetermined maximumoperating temperature, and to set the flow rate of the fifth coolingliquid such that the temperature of the second electronic device doesnot exceed a second predetermined maximum operating temperature.

Alternatively, the cooled electronic system further comprises a fourthheat transfer device comprising a fifth channel for receiving the fifthcooling liquid from the fourth channel, and a sixth channel forreceiving a sixth cooling liquid for coupling to a heat sink, the secondheat transfer device being configured to transfer heat between the fifthchannel and the sixth channel.

In an eighth aspect, there may be found in a method of filling theinterior a container for an electronic device with a cooling liquid, themethod comprising: adapting the container to receive the cooling liquidby at least one of: heating the container to a filling temperature; andreducing the pressure in the interior of the container; filling thecontainer with the cooling liquid; and sealing the container to preventleakage of the cooling liquid.

By adapting the container and liquid to a filling temperature orpressure and filling the container with the liquid at this temperatureor pressure, the volume of the interior space of the container that isfilled with liquid under operating conditions is increased, withoutsignificantly increasing the pressure within the container. Excessivepressure may result in damage to the container or the electronic device.

Optionally, the method further comprises: heating the cooling liquid tothe filling temperature; and cooling the sealed container and coolingliquid to an operating temperature.

In some embodiments, the step of filling the container is carried outbefore the step of adapting the container. Optionally, the step ofadapting the container comprises operating the electronic device.

In a ninth aspect, there is provided a method of filling the interior acontainer for an electronic device with a cooling liquid, the methodcomprising: heating the cooling liquid to a filling temperature; fillingthe container with the heating cooling liquid; sealing the container toprevent leakage of the cooling liquid; and cooling the sealed containerand cooling liquid to an operating temperature. This is an alternativeto the eight aspect and the eighth and ninth aspects can also optionallybe combined.

In either or both of the eighth and ninth aspects, there are a number ofpreferable or optional features. Advantageously, the filling temperatureis selected such that gases dissolved in the cooling liquid are removedfrom the cooling liquid. Hence, air, moisture and other dissolved gasesare removed from the interior of the container. The need for a desiccantin the container is reduced or avoided.

Preferably, the filling temperature is selected on the basis of themaximum operating temperature of the electronic device. Beneficially,the filling temperature is selected to be equal to or greater than themaximum operating temperature of the electronic device.

In the preferred embodiment, the step of filling the container iscarried out such that all air in the container is displaced.

There is also provided a cooling apparatus configured to use a liquidcoolant for carrying heat removed from an electronic device. The coolingapparatus includes a heat exchanger arrangement, configured to receivethe liquid coolant, to transfer heat from a first portion of the liquidcoolant to a first heat sink and to transfer heat from a second portionof the liquid coolant to a second heat sink. The first and second heatsinks may be isolated from one another. Moreover, there is provided amethod of operating a cooling system that uses a liquid coolant forcarrying heat removed from an electronic device, in which a firstportion of the liquid coolant may be received at a heat exchangerarrangement, so as to transfer heat from the first portion of the liquidcoolant to a first heat sink. A second, separate portion of the liquidcoolant may be received at the heat exchanger arrangement, so as totransfer heat from the second portion of the liquid coolant to a secondheat sink. The first and second heat sinks may be isolated from oneanother.

A method of manufacturing a cooling system that uses a liquid coolantfor carrying heat removed from an electronic device may includeproviding a first heat exchanger for receiving a first portion of theliquid coolant, so as to transfer heat from the first portion of theliquid coolant to a first heat sink; and providing a second heatexchanger in parallel with the first heat exchanger for receiving asecond, separate portion of the liquid coolant, so as to transfer heatfrom the second portion of the liquid coolant to a second heat sink. Thefirst and second heat sinks may be isolated from one another.

In a further aspects, there may be provided a cooling apparatus whichincludes an interfacing component configured to receive an electronicdevice and defining a thermal interface through which heat generated bythe electronic device can be transferred away from it; and a heatexchanger arrangement, coupled to the thermal interface and configuredto transfer heat generated by the electronic device via the thermalinterface to a first heat sink and a second heat sink. The first andsecond heat sinks may be isolated from one another.

Thus, redundancy may be provided at the level of a module, which may bea single container or mounting used for cooling one or more components.Each component may be a single electronic component or it may be adevice comprising a plurality of electronic components (such as acircuit board) or a plurality of such devices.

Advantageously, in the event that a part of the heat exchangerarrangement is unable to transfer heat to the first heat sink, the heatexchanger arrangement may be configured to transfer at least some (orall) of the heat that would have been transferred to the first heat sinkto the second heat sink. Thus, if the part of the heat exchangerarrangement transferring heat to one heat sink fails, the heat exchangerarrangement may be configured to transfer a greater proportion of heatto the other heat sink. This may act to provide redundancy.

In some embodiments, the heat exchanger arrangement is configured totransfer an approximately equal proportion of the heat received from theelectronic device to the first heat sink as the second heat sink. Inother words, the heat exchanger arrangement may divide the heat outputbetween the two heat sinks roughly equally, such that the two share theload. However, the ratio of heat transfer may be controlled or may beset differently (for example, 45%:65%, 40%:60%, 5%:95%). The capacity ofthe first heat sink and the second heat sink may be the same in manyembodiments.

In some embodiments, the interfacing component includes a housing thatdefines at least part of a volume in which a coolant and at least partof the electronic device can be located, such that the coolant canprovide the thermal interface by convection. The heat exchangerarrangement may then include a first heat exchanger, having a partlocated in a first part of the volume and configured to transfer heatfrom a first portion of the coolant to the first heat sink; and a secondheat exchanger, having a part located in a second part of the volume andarranged in parallel with the first heat exchanger, the second heatexchanger being configured to transfer heat from a second portion of thecoolant to the second heat sink.

Thus, a sealable module that is similar to the module described abovemay be provided, but with a second, parallel heat exchanger. Thisparallel heat exchanger provides redundancy, allowing cooling tocontinue even if the first heat exchanger fails for any reason.Moreover, it increases the efficiency of cooling.

In some configurations it may be preferred that the first heat exchangerinclude a first conduction surface that cooperates with the housing soas to define at least part of the volume. In other words, the sealablemodule may be formed as a container, a part of the container walls beingdefined by the housing and another part of the container walls beingdefined by the first conduction surface.

In embodiments, the first heat exchanger further defines a first channelfor receiving a first outer liquid coolant. Then, the first conductionsurface may separate the volume and the first channel to allowconduction of heat between the volume and the first channel through thefirst conduction surface. Thus, the first heat sink is provided by thefirst outer liquid coolant that carries heat from the liquid coolantwithin the volume away from the sealable module via conduction throughthe first conduction surface.

Advantageously, the cooling apparatus also may include the electronicdevice. Then, at least a portion of the first conduction surface orhousing may be shaped in conformity with the shape of the electronicdevice. This conformity of shape may further improve efficiency.

The second heat exchanger may be formed in a similar way to the firstheat exchanger. Thus, the second heat exchanger may include a secondconduction surface that cooperates with the housing (and optionally, thefirst conduction surface) so as to define at least part of the volume.Then, the second heat exchanger may further define a second channel forreceiving a second outer liquid coolant. In this case, the secondconduction surface may separate the volume and a second channel to allowconduction of heat between the volume and second channel through thesecond conduction surface. In this way, the first and second outerliquid coolants may be isolated from one another. This may achieve theisolation of the first and second heat sinks. Advantageously, this mayalso improve the flexibility, efficiency and robustness of the first andsecond heat exchangers.

Where the cooling apparatus includes the electronic device, at least aportion of the second conduction surface or housing is optionally shapedin conformity with a shape of the electronic device. This may depend onthe configuration of the component or components, but also on theconfiguration of the first and second heat exchangers and particularlythe first and second conduction surfaces.

The electronic device may take a large number of different forms. In apreferred embodiment, the electronic device includes a circuit board.This may then define a substantially planar form for the electronicdevice.

Whether or not the electronic device is planar, it may have an axis ofelongation. This especially applies when the electronic device includesa circuit board, but it can also be the case where the electronic deviceis a discrete package or a combination of discrete packages not mountedon a circuit board. While the electronic device has an axis ofelongation, the first conduction surface may have a respective axis ofelongation that is substantially parallel to the axis of elongation ofthe at least one electronic component. Additionally or alternatively,the second conduction surface may have an axis of elongation that issubstantially parallel to the axis of elongation of the at least oneelectronic component. In a preferred embodiment, both the first andsecond conduction surfaces have parallel axes of elongation.

The first conduction surface may have a substantially planar form.Additionally or alternatively, the second conduction surface may have asubstantially planar form. Where the electronic device and the firstconduction surface both have a substantially planar form, the planes ofthe first conduction surface may be substantially parallel to the planeof the electronic device. Additionally or alternatively, the electronicdevice and the second conduction surface may both have a substantiallyplanar form and the plane of the second conduction surface may besubstantially parallel to the plane of the electronic device.

In some embodiments, the housing and the first and second conductionsurfaces define the volume. Thus, a container with integrated heatexchangers can be provided. Optionally, the housing, the first andsecond conduction surfaces and the electronic device define the volume.Then, the electronic device itself forms an integral part of the moduleouter container. In particular, the volume may further includes awicking material such that a vapour chamber is formed.

The coolant may preferably include a fluid. In many embodiments, thecoolant includes a liquid (defined at standard or operating temperatureand pressure). In embodiments, the cooling apparatus further includesthe coolant.

Where the heat exchanger arrangement or the interfacing componentincludes a conduction surface, the heat exchanger arrangement mayfurther include a first piping system, arranged to carry a firstcoolant, to receive heat directly from the conduction surface and totransfer heat through the conduction surface to the first coolant; and asecond piping system, arranged in parallel with the first piping systemto carry a second coolant, to receive heat directly from the conductionsurface and to transfer heat through the conduction surface to thesecond coolant. The first and second coolants may be isolated from oneanother. In these embodiments, the heat exchanger arrangement may beprovided by a single cold plate with multiple coolant channels, each ofwhich is isolated from the others. This provides redundancy by use of asingle heat exchanger (the cold plate), whilst still providing twoseparate heat sinks.

In embodiments, the first piping system and second piping system arearranged in parallel in a three-dimensional crossing arrangement, suchas in a helical arrangement. In other words, the two channels cross (orspiral) to receive roughly equal proportions of the heat and avoidplacing too great a load on one heat sink under normal operation. Insome embodiments, the interfacing component preferably includes theconduction surface that is configured for attachment to the electronicdevice, such that the thermal interface is provided by conduction. Thus,the cold plate may be affixed directly to a surface of the electronicdevice (such as a circuit board). Some thermally conductive material mayinterpose between the electronic device and the conduction surface,although the conduction surface is still physically attached to theelectronic device through this material.

In some embodiments, the interfacing component includes a housing thatdefines at least part of a volume in which a third coolant and at leastpart of the electronic device can be located. This may mean that thethird coolant can provide the thermal interface by convection to theconduction surface that forms part of the heat exchanger arrangement.Then, the third coolant may be isolated from the first and secondcoolants.

In other aspects, there may be provided a cooling system including thecooling apparatus as described herein, as well as a piping system,configured to carry a first outer coolant for acting as the first heatsink of the first heat exchanger arrangement and to carry a second outercoolant for acting as the second heat sink of the heat exchangerarrangement. Then, the first and second outer coolants may be isolatedfrom one another. The piping system may then be configured to allowmultiple modules to be connected with a series or parallel connection(or combination of the two) to share the first outer coolant, secondouter coolant or both.

It may be advantageous for the cooling system to include a second heatexchanger arrangement, configured to receive a first outer coolant and asecond outer coolant. Then, the second heat exchanger arrangement may beconfigured to transfer heat from the first and second outer coolants toa common output heat sink. The heat exchanging arrangement may includeone or more than one heat exchanger. It will therefore be understoodthat the first and second heat sinks of the first and second heatexchangers are isolated from one another, but these two heat sinks mayhave their own heat sinks and these heat sinks need not be isolated fromone another.

In further aspects, there may be provided a method of operating acooling system that includes coupling an electronic device to aninterfacing component that defines a thermal interface through whichheat generated by the electronic device can be transferred away from it;operating the electronic device to generate heat that is transferred tothe thermal interface; receiving heat generated by the electronic deviceat a heat exchanger arrangement via the thermal interface; andtransferring heat received at the heat exchanger arrangement to a firstheat sink and a second heat sink. The first and second heat sinks may beisolated from one another.

It will be understood that method features corresponding with thestructural, apparatus features described above may optionally beprovided in conjunction with the method. Some of these are nowexplicitly defined below.

In some configurations, such as in the event that a part of the heatexchanger arrangement is unable to transfer heat to the first heat sink,at least some of the heat that would have been transferred to the firstheat sink may be transferred to the second heat sink. Optionally, thestep of transferring heat may include transferring an approximatelyequal proportion of the heat received from the electronic device to thefirst heat sink as the second heat sink.

Optionally, an interfacing component may include a housing that definesat least part of a volume. Then, a coupling technique as disclosedherein may include locating at least part of the electronic component inthe volume together with a coolant, such as a liquid coolant, such thatthe coolant can provide the thermal interface by convection. Here, astep of receiving heat may include receiving a first portion of thecoolant at a first heat exchanger located in a first part of the volume,so as to transfer heat from the first portion of the coolant to thefirst heat sink; and receiving a second, separate portion of the coolantat a second heat exchanger arranged in parallel with the first heatexchanger and located in a second part of the volume, so as to transferheat from the second portion of the coolant to the second heat sink.

In order to receive the first portion of the coolant at the first heatexchanger, heat received from the first portion of the coolant may betransferred to a first outer coolant. Additionally or alternatively, inorder to receive the second portion of the coolant at the second heatexchanger, heat received from the second portion of the coolant may betransferred to a second outer coolant. Where the first and second outercoolants are used, these may be isolated from one another.Advantageously, the first and second outer coolants also may be liquids.

Such a method may further include receiving the first and second outercoolants at a heat exchanging arrangement; and transferring heat fromthe first and second outer coolants to a common output heat sink usingthe heat exchanging arrangement.

In yet further aspects, there may be provided a method of manufacturinga cooling system, which includes coupling an electronic device to aninterfacing component that defines a thermal interface through whichheat generated by the electronic device can be transferred away from it;mounting a heat exchanger arrangement on the interfacing component inorder to transfer heat from the electronic device to the heat exchangerarrangement via the thermal interface; and configuring the heatexchanger arrangement so as to transfer heat received from theelectronic device to a first heat sink and a second heat sink. The firstand second heat sinks are isolated from one another. Optionally, themethod may further include filling the volume with the cooling liquid.

Again, it will be understood that this method may include optionalmethod steps corresponding with any one or more of the apparatusfeatures defined herein.

A coupling technique as disclosed herein may include housing anelectronic device in a volume defined by a sealable module, such that aliquid coolant may be added to the volume for removing heat generated bythe electronic device. Then, a heat exchanger arrangement may be mountedby mounting a first heat exchanger in a first part of the volume so thatit can receive a first portion of the liquid coolant and transfer heatfrom the first portion of the liquid coolant to a first heat sink; andmounting a second heat exchanger in parallel with the first heatexchanger in a second part of the volume so that it can receive asecond, separate portion of the liquid coolant and transfer heat fromthe second portion of the liquid coolant to a second heat sink.Optionally, the method may further include filling the volume with thecooling liquid.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may be put into practice in various ways, a number ofwhich will now be described by way of example only and with reference tothe accompanying drawings in which:

FIG. 1 shows a simplified front view of an equipment rack containingmultiple cooling units, each cooling unit comprising two sealablemodules;

FIG. 2 is a simplified cross-sectional side view of the equipment rackshown in FIG. 1;

FIG. 3 illustrates a cooling unit, as shown in FIG. 1, fitted with acover;

FIG. 4 depicts a cooling unit with its cover removed, showing twosealable modules;

FIG. 5 is a cross-sectional exploded view of a sealable modulecomprising a heat generating electronic component according to anembodiment;

FIG. 6 is a cross-sectional view of the upper part of the sealablemodule of FIG. 5;

FIG. 7 shows the sealable module face of an embodiment of a cold plate,for use with the sealable module of FIG. 5;

FIG. 7A shows an embodiment of the opposite face of a cold plate fromthat shown in FIG. 7, showing liquid flow channels for a second coolingliquid, for use with the sealable module of FIG. 5;

FIG. 7B shows an alternative embodiment of the face of the cold plate 60shown in FIG. 7A;

FIG. 8 is a schematic view of a three-stage cooling system comprising asingle cooling unit;

FIG. 9 is a block diagram showing a three-stage cooling system withmultiple cooling units;

FIG. 10 shows a monitoring and control system for use with thethree-stage cooling system of FIG. 9;

FIG. 11A shows a first side view of a second embodiment of a sealablemodule;

FIG. 11B shows a second side view of the embodiment shown in FIG. 11A;

FIG. 11C shows a more detailed view of the embodiment shown in FIG. 11Aand FIG. 11B;

FIG. 12 depicts a schematic diagram of a cooling system withcabinet-level redundancy;

FIG. 13 is a schematic diagram of a cooling apparatus with additionalredundancy;

FIG. 14 is a cross-sectional exploded view of an embodiment of asealable module comprising a heat generating electronic component inaccordance with the schematic diagram of FIG. 13;

FIG. 15 is a schematic diagram of a cooling apparatus with additionalredundancy as an alternative to that shown in FIGS. 13 and 14;

FIG. 16 is a schematic diagram of a cooling apparatus with additionalredundancy as an alternative to that shown in FIGS. 13, 14 and 15; and

FIG. 17 is a schematic diagram of a cooling apparatus with additionalredundancy as an alternative to that shown in FIGS. 13, 14, 15 and 16.

SPECIFIC DESCRIPTION OF PREFERRED EMBODIMENTS

Referring first to FIG. 5, there is shown a cross-sectional explodedview of a sealable module 41 comprising a heat generating electroniccomponent 69 according to an embodiment. The sealable module comprises:a housing 81; a finned conduction surface 71 forming part of a coldplate 60; a container volume, defined after assembly of the componentsby the housing 81 and conduction surface 71 and filled with a firstcooling liquid (not shown); liquid flow channels 61 adjacent theconduction surface 71; small electronic component 68; large electroniccomponent 69; and memory module 76.

The sealable module further comprises: an electronic circuit board 75;mounting pillars 63 for the electronic circuit board 75; a componentheat sink 70 attached to the large electronic component 69; screws 80 toattach the mounting pillars 63 to the conduction surface 71; a coverplate 78 for the side of the cold plate opposite to the fins on theconduction surface; insulation 73 for the housing 81; first sealinggasket 62; second sealing gasket 64; screws 79 to hold the components ofthe cold plate assembly together; and pin-fin projections 65 on theconduction surface 71. The insulation 73 can also serve as a protectivemembrane between the housing 81 and the circuit board 75.

The cold plate 60 is fabricated with two faces, each with a separatefunction. Conduction surface 71 is a pin-finned plate, forming one faceof the cold plate. A housing 81 is attached to the pin-finned plate 71,in such a way as to provide an internal space for an electronic circuitboard 75, the pins of the cold plate and a first cooling liquid (notshown). A gasket 62 ensures that the assembled capsule is substantiallysealed against liquid loss or ingress of air. The pin-finned plate iseffectively the lid of the assembled capsule.

The electronic circuit board 75 carrying components to be cooled isattached to the cold plate 60 by mounting pillars 63, so as to suspendthe board from the cold plate 60, allowing accurate alignment of thefins of the cold plate with components on the board, prior to attachingthe housing 81.

Alternatively, the board may be attached to housing 81 by mountingbosses extending from the housing 81 or equivalent means.

The fins 65 of the pin-finned conduction surface 71 normally face thecomponent side of the circuit board 75. In some cases, components ofsignificant size may be present on both sides of the board. The housingmay then be contoured around the components on the side of the boardopposite the pin-finned conduction surface 71, in order to improve flowof the cooling liquid and reduce the amount of cooling liquid needed.

In the embodiment shown in FIG. 5, the cold plate is fabricated in asingle part, with two separately formed faces. Alternatively, the platemay be manufactured in two parts: a pin-finned plate whose opposite faceis flat; and a plate with channels 61 for liquid flow, again with a flatopposite face; the two flat faces being joined on assembly. The coldplate assembly 60 has a surface opposite to the pin-finned conductionsurface into which channels whose cross section is shown at 61 aremanufactured.

The component side of the electronic circuit board 75 faces the fins 65on the conduction surface 71. A small gap between the ends of the finsand the components is provided. The fins have an elongated cross-sectionand the height of the fins varies, so as to maintain a small gap betweenthe variously sized components on the electronics circuit board and thetops of the fins. The faces of the electronic components 68 and 69 andthe edge of the memory module 76 project by different amounts from thesurface of the board. Small component 68 has a relatively low profileand large component 69 has a much deeper profile and the correspondingfins 65 have accordingly different heights. The height of the pin-finsand the gap between components on the circuit board 75 and the top ofthe pin-fins are arranged to be as small as possible, consistent withthe requirement for efficient cooling. This arrangement reduces thetotal quantity of cooling liquid needed in the capsule and improvespacking density of cooling units.

When the system is in operation, heat generated by the components 68, 69and memory module 76 on the circuit board is transferred to the coolingliquid (not shown), initially by local conduction and then, as theheated liquid expands and becomes buoyant, by convection. The convectingliquid quickly comes into contact with the fins 65 and other surfaces ofthe conduction surface 71.

Heat from the fins 65 of the conduction surface 71 is conducted to thecirculating second liquid that flows via channels 61, so as to cool theconduction surface 71 and thus cooling liquid.

Components on circuit board 75 that generate the largest amount of heatare typically microprocessors. In this case, cooling efficiency for suchcomponents may be improved by additionally fitting a finned componentheat sink 70, in direct physical and thermal contact with the component,whose fins may interleave with the array of fins on the cold plate. Thefins of the component heat sink may be at least partially in physicalcontact with the pin-fins of the cold plate. Gaps remain through whichthe first cooling liquid can flow.

An additional insulating layer 73 is provided, preferably on the insideof the housing 81 of the sealable module 41. Additional insulation mayalso be added to the exterior of the cold plate cover 78 and to theedges of the cold plate 60. The insulation reduces local heat loss intothe atmosphere, which can be significant in large installations withmany electronic circuit boards, causing room temperature to rise toundesirable levels.

Electronic circuit board 75 may carry major components only on one sideor the other side may carry only small components that generate lowamounts of heat in operation. The operation of the sealable module 41may then be improved by excluding liquid flow adjacent to thenon-component side of the circuit board 75. The insulation 73 may thenact as a flexible protective membrane between the circuit board 75 andthe housing 81, that can accommodate to the shape of the smallcomponents that may be mounted on this side of the board. Alternatively,a separate protective membrane (not shown) could be provided between thehousing 81 and the electronic circuit board 75. Convective flow of thecooling liquid is then concentrated in the space between the maincomponent side of the circuit board 75 and the conduction surface 71.

Application of the invention is not limited to computing systems.However, since computing systems generate significant heat, they canbenefit from improved cooling. In such systems, one or moremicroprocessors and several other digital and analogue devices such asmemory chips (RAM, ROM, PROM, EEPROM and similar devices), specialisedintegrated circuits (ASICs) and a range of associated active and passivecomponents are typically mounted on a circuit board, whose function isto act as a major part of a computing system. Although electroniccomponents can be mounted on both sides of the circuit board, it is moreusual to mount at least the bulky components on one side only. Otherdevices are connected to the circuit board by cables, optical means orwireless transmission, the whole forming a computer, computer system orserver.

Heat is generated by the various components but, typically, themicroprocessor is the highest heat-generating component. An optimallydesigned cooling system removes sufficient heat from each component tokeep it within its designed operating temperature range but no more thanthat. Devices that generate less heat need less cooling than those thatgenerate larger amounts. Cooling below a level necessary forsatisfactory operation will normally consume unnecessary additionalenergy and is therefore less than optimally efficient.

Moreover, packing density of components on computer boards is determinedpartly by the traditional size of computer housings and the assumptionthat air-cooling may be employed. In large systems, especially servercentres, increasing the packing density of components to reduce overallspace occupied is desirable. At the same time, heat generated will beconcentrated in a smaller space and needs more effective means ofremoval. Improving heat removal may enable component packing density tobe increased and, more particularly, allow processing power per unitvolume to increase.

Referring now to FIG. 6, there is shown a cross-sectional view of theupper part of the sealable module 41 of FIG. 5. Where the same featuresare shown, identical reference numerals are used. The depth of themodule is exaggerated to clarify details of some features that wouldotherwise be difficult to illustrate. FIG. 6 additionally shows: firstcooling liquid 66; a filling inlet 44; seal 43; a pressure relief device45; fastener82 for receiving assembly screw 79; sealing gasket 62;baffle plate 74; and capacitive rods 72. The filling inlet 44 isintended for receiving the first cooling liquid 66 and has a seal 43 toprevent liquid loss once the sealable module 41 is filled. A pressurerelief device 45 allows escape of liquid under extreme conditions,outside a normal range of: temperature; pressure; or both.

Normally, larger components that generate the highest amount of heat arelocated towards the lower part of the module 41. In some circumstances,the heated liquid 66 may rise in a narrow convection plume that couldoverheat components located higher in the module. Baffle plate 74 maythen be fitted to deflect the hot plume of liquid away from thecomponents in the upper part of the module and aid mixing with coolerliquid for re-circulation within the module. Similarly, where two ormore processors that generate large amounts of heat are fitted tocircuit board 75, it is possible that one processor is above the otherand receives heated liquid from the lower mounted processor. A baffleplate 74 or a similar or equivalent means of passively deflecting hotliquid may then be fitted in the lower part of the module.

Typically, cooling liquid 66 is a volatile substance. Small leaks fromthe module may be difficult to detect, since the liquid may evaporatebefore being observed. Capacitive rods 72 may then be fitted in theupper part of the module 41, normally immersed in liquid 66 andconnected to an external detector (not shown) that measures thecapacitance. In the event that the liquid leaks from the module, thelevel of liquid 66 will drop and the capacitance will alter. If thecapacitance alters appreciably, an alarm can thus be generated,indicating excessive loss of liquid from the module. The capacitive rods72, the connections thereto and a detection circuit may alternatively bebuilt in to a customised electronic circuit board, thus simplifyingassembly of the complete capsule and removing the possibility of leakageof liquid from the entry point of the capacitive rods 72 into thehousing 81.

An alternative means of monitoring the liquid level is a smalltransparent window (not shown) that may also be fitted to the upper partof housing 81. This window allows direct observation of the level ofliquid 66 within the module.

Referring now to FIG. 7, there is shown one face of an embodiment of acold plate 60, for use with the sealable module 41 of FIG. 5. This faceprovides a pin-finned plate that attaches to housing 81 as shown in FIG.5 to form a sealable first stage cooling module. FIG. 7A shows theopposite face of the embodiment of the cold plate.

The face of cold place 60 shown in FIG. 7 comprises: conduction surface71; first pin-fin 96; second pin-fin 97; a channel 87 for a sealinggasket; and holes 88 for assembly screws to attach the housing 81 ofFIG. 5. The pin-fins 96 and 97 form projections from the conductionsurface 71.

An inlet 84 and outlet 83 for the second cooling liquid are visible inFIG. 7 but connect to the other side of the cold plate. Mounting lugs 90for the complete module also support the liquid inlet and outlet pipes83, 84. The pin-fins 96 are of greater height than pin-fins 97. Theillustrated pin-fins are examples and the actual layout and size ofpin-fins may vary according to the shape and heat-generatingcharacteristics of the components to be cooled.

Turning now to FIG. 7A, there is shown an embodiment of the oppositeface of the cold plate 60 to that illustrated in FIG. 7, for use withthe sealable module 41 of FIG. 5. Flow directing fingers 85, 89 createchannels 91 in the cold plate 60 that form a continuous winding patternwithin the boundary of the cold plate 60 and join via holes 92 to tubesthat emerge as inlet connector 84 and outlet connector 83 for the secondcooling liquid.

The flat part of this side of the cold plate 60 is the opposite side ofconduction surfaces 71 shown in FIG. 7. Cover 78 of FIG. 5 is attachedto the cold plate by screws that align with holes 88 and is sealed by agasket that fits in channel 87, so as to enclose the channels 91, thuscreating a winding channel arrangement within the assembly. Since theassembled arrangement of channels requires no maintenance, an improvedassembly may be constructed by welding the cover 78 directly to the coldplate 60 base part. Adhesive or other techniques for fixing the coverpermanently can alternatively be used. Gasket 64 and assembly screws arethen not required and the potential for leakage of the second coolingliquid is reduced.

Alternatively, the single winding channel arrangement may be branched toform two or more channels, with common input and output connections forthe second cooling liquid. The two or more channels may be of similar ordifferent dimensions, and may be winding or straight, so as to provide adifferent flow rate of second cooling liquid over different parts of thecold plate.

Turning to FIG. 7B, there is shown an alternative embodiment of the faceof the cold plate 60 shown in FIG. 7A. This illustrates one possiblearrangement with several channels, aligned generally in a verticalorientation. The channels are bounded by flow directing fingers 93. Theflow of liquid entering via inlet 84 is divided amongst the channels;the flow rate in each channel being dependent on the width of thechannel, the shape of the entrance to the channel and the location ofthe liquid entry hole 92.

Thus, it is possible to arrange the assembly so as to provide similar ordifferent liquid flow rates over different areas and thus different heattransfer rates from different parts of the plate. Electronic componentsadjacent to the conduction surface on the other side of plate 60 fromthe channels may generate different amounts of heat. With higher levelsof heat production, this may advantageously be made to correspond withthe areas with higher rates of heat transfer.

A degree of adjustment may, optionally, be included in the assembly. Oneor more adjustable baffle plates 94 attached by locking screws 95 may bepositioned so as to direct the flow of liquid more towards or away fromone of the channels. The baffle plate may be adjusted by slackeningscrew 95, rotating the baffle plate to a new position and thenre-tightening the screw. In this example, the adjustment is made beforethe cover is attached, although it would be readily possible to addmeans of adjusting the baffle plate from the exterior of the assembledunit.

Mounting lugs 90 for the complete assembly also support the inletconnector 84 and outlet connector 83.

The material used for the cold plate 60 and conduction surface 71 ischosen to be a good conductor of heat, typically a metal. For ease offabrication and lower cost in quantity production, a plastic materialcould be employed with lower but still adequate heat conductionproperties. The channels 91 formed within the cold plate 60 are used tocarry a second cooling liquid that circulates through the cold plate andthen outside to carry heat away to further cooling stages of the system.

The sealable modules 41 provide a first stage of cooling and form partof cooling units, each cooling unit carrying one or more modules. Atleast one and typically many first-stage cooling units are deployed in asystem. The cooling units may be fitted into any convenient housing but,where large numbers are used in a system, conventional equipment rackswould normally be used.

In FIG. 4, there is shown a cooling unit with its cover removed, showingtwo sealable modules. Where the same features are shown in FIG. 4 as inFIGS. 6 and 7, identical reference numerals are used. In addition isshown: frame 31; locking tabs 32; data transfer cable 46; power cable47; first seal for cable entry 50; second seal for cable entry 51; andtest and monitoring panel 52.

Essentially, FIG. 4 shows the sealable part of each cooling module 41,the remaining part being on the opposite side of the module, separatedby the internal cold plate and conduction surface of FIG. 5. The coolingunit comprises a frame 31, which supports cooling modules 41 and variousliquid and electrical interconnections. The front panel of the frame 31carries a test and monitoring panel 52 and locking tabs 32, which canrotate about a hinge so as to lock the unit in place in an equipmentrack.

Two sealed modules 41 are shown. The housing 81 for each sealable module41 is made of plastic or equivalent material, chosen to be an electricalinsulator, and to have heat insulating properties, as well as notreacting with the cooling liquid used in the module. Housing 81 is helddown by fasteners 56. Cables 46, 47, carrying electrical power andsupporting bi-directional data transmission enter the capsule via entrypoints 50 and 51, sealed to prevent the escape of liquid or the ingressof air. Cables 46, 47 terminate in respective connectors (not shown) atthe rear of the cooling unit.

The cold plate side of each module (not shown in FIG. 4) is cooled by acirculating second liquid. Each of the two sealable modules 41 isconnected via an inlet pipe 53 for the second cooling liquid, to aliquid flow splitter 55. The liquid flow splitter 55 has two outputs anda common input, connected by a further pipe 58. This splits the flow ofsecond cooling liquid between modules 41 and the common input isconnected by a further pipe 58 to liquid input connector 12 on the rearpanel of the cooling unit. Similarly, a liquid outlet pipe 54 from eachof the two sealable modules carries liquid to a flow-combining unit 57,the common output of which is connected by a further pipe 59 to liquidoutput connector 11 on the rear panel of the cooling unit.

The assembled sealable module 41 is partly or wholly filled with a firstcooling liquid 66 via the filling inlet 44 and then sealed with sealingdevice 43. The filling procedure may take place during factory assemblyor during field installation of cooling modules.

During filling, the sealable module 41 is partly filled with liquid, theremaining space being occupied by a mixture of its vapour and someresidual air. One method of achieving this is by heating the liquid andthe module to a filling temperature (T_(fill)), selected to be wellabove ambient temperature and approximately the same as the maximumoperating temperature of the system (T_(max)). The maximum storagetemperature of the electronic components is typically much higher thanmaximum operating temperature, so that T_(fill) can either be below,equal to or above T_(max), the highest envisaged operating temperatureof the system.

Liquid is then added to displace most of the air within the sealablemodule 41, such that the level of liquid is sufficient substantially toimmerse all the components to be cooled. The sealable module 41 is thensealed with sealing device 43 to prevent liquid escape and ingress offurther air. The sealable module 41 is then allowed to cool to roomtemperature. The liquid contracts and leaves a space, occupied by liquidvapour and air mixture. The filling procedure may take place in two ormore steps, allowing time for liquid that has been added to the sealablemodule 41 in one step to cool partly before adding more.

At ambient temperature, the vapour and air in the filled and sealedmodule is thus below atmospheric pressure. This can rise duringoperation so as be equal to or moderately higher than externalatmospheric pressure. A module filled at ambient temperature and thenimmediately sealed would, during operation and heating to T_(max), besubjected to much higher and potentially damaging internal pressure thanone filled by the method described.

The method may, if desired, be extended to exclude all air from theliquid by filling the module completely at T_(fill) and choosingT_(fill) to be higher than T_(max), so that, on cooling, the remainingspace above the liquid is filled only with vapour from the liquid at alow pressure below atmospheric. Heating the liquid has the additionalbenefit of driving out some or most of the dissolved gaseouscontaminants that may be present in the untreated liquid.

A further alternative method of filling is to add liquid via fillinginlet 44 to the module 41 when both are at ambient temperature, so as tofill to a pre-determined level, sufficient to immerse substantially theelectronic components to be cooled. Before sealing with seal 43, themodule is then connected to its power supplies and the electroniccomponents set into operation in such a way as to elevate the liquidtemperature to or close to T_(max). The module is then sealed,disconnected from its power supplies and left to cool.

Yet another method of filling is under vacuum, such that liquid can beadded to the module at ambient temperature, whilst all or most air isexcluded. One way of achieving this is to fit a valve and a means ofconnecting a tube for air to be pumped out and liquid to enter tosealing device 43. The tube opens the valve when connected but releasesthe valve when removed and allows it to seal the module automatically.The tube has a tee connection, one arm of which is connected to a vacuumpump via a further closable valve. The other arm of the tee connectionconnects via a flexible tube to a container filled with a pre-determinedquantity of the filling liquid. Initially, the container is held at alevel where the liquid is below the level of the filling inlet. The endsof the tube make an airtight seal with the module and containerrespectively. The valve to the vacuum pump is opened and most of the airis pumped out of the module and liquid container. The valve to thevacuum pump is then closed and the container is raised so as to be abovethe level of the filling inlet. Liquid then flows into the module so asto fill the available space to a pre-determined level and immerse thecomponents to be cooled. Finally, the tube with tee connection iswithdrawn from the module and the filling inlet valve sealsautomatically against ingress of air or loss of liquid. The module hasthus been filled with liquid to a predetermined level, leaving a vapouror air space at low pressure, below atmospheric.

An alternative method that is also useful in field filling is to fillwith a cool or warm liquid, since there is increased danger of spillinghot liquid. In this case, an air gap is always left above the liquid toallow for expansion. Liquid may be factory prepared to remove dissolvedgases and is then stored in sealed containers. The interior space of thesealable module 41 is filled with dry air and the sealing device fitted.When filling the sealable module 41 with liquid, the sealing device 43is removed, a specified amount liquid is poured into the sealable module41 via the filling inlet 44 and the sealing device 43 is thenimmediately refitted. The specified amount of liquid added is sufficientfor effective cooling but leaves a remaining space filled with air forexpansion of the liquid at temperatures up to T_(max), the highestenvisaged operating temperature of the system.

At temperatures below the lowest envisaged room temperature, the liquidmay contract further and the electronic circuit board 75 may no longerbe fully immersed in liquid. This is envisaged to occur when the moduleis inactive, in storage or being transported, for example by air, whenlow external pressure and temperature conditions may occur. The seal 62between the housing and cold plate is intended to withstand temperatureand pressure variations between the limits envisaged for inactivity,storage and transportation and the conditions at T_(max).

Above T_(max), the system would be outside its design temperature range.Although higher temperatures are very unlikely, the pressure reliefvalve device 45 allows escape of liquid that has exceeded T_(max), thetemperature at which the liquid fills or is close to filling theavailable space inside the module. The pressure relief device may becombined with the seal 43 for the filling inlet 44.

The first cooling liquid 66 is chosen on the basis of a number ofdesirable characteristics. It should not significantly affect theperformance of the electronic circuit board 75 or the transmission ofinformation between the circuit board 75 and other external devices. Itshould not be corrosive to any component of the cooling module, remainliquid at all operating, storage and transportation temperatures, havesufficiently good specific heat capacity, in order to carry heat awayfrom the electronic components as efficiently as possible, have a highenough coefficient of expansion and low enough viscosity to aid rapidconvection, be low-cost, be safe to use and be non-hazardous in case ofleakage.

One example of a suitable first cooling liquid 66 is a hydrofluoroetherchemical. This has all the desirable characteristics, including a highcoefficient of expansion and sufficiently high specific heat capacity toprovide high mass-flow rate and rapid convection when heated, thuscarrying heat quickly away from the hot components.

In FIG. 3, there is illustrated the cooling unit of FIG. 4, on whichframe 31 is fitted with a cover 33 to form an assembled cooling unit 2.Cooling unit 2 further comprises: first data connector 27; second dataconnector 28; first power connector 29; second power connector 30; andfront panel locking tabs 32.

When housed in a standard equipment rack, each cold plate 60, within itsrespective sealable module 41, is commonly in the vertical plane. Eachcold plate 60 carries liquid from inlet 12 also associated with a pairof sealable modules 41 inside the cooling unit 2.

The cover 33 is held in place by screws 34 or equivalent fixings,protects the sealable modules 41 and other internal parts of the coolingunit 2 and gives additional EMC protection. The cover additionallycompletes an external rectangular box shape that is convenient forsliding into and out of a shelf in a rack for installation, repair orreplacement.

Electrical connections are also made at the rear of the module, forpower 29,30 and data transfer 27, 28. Standard connectors may beemployed to allow connection and disconnection of the module forinstallation and removal.

Reference is now made to FIG. 1, in which there is shown a simplifiedfront view of an equipment rack containing multiple cooling units.Equipment rack 1 comprises: cooling unit 2; additional equipment shelves3; AC power unit 4, which is commonly air-cooled; and cold plate 5 toprovide additional cooling for the AC power unit 4. The AC power unit 4may alternatively be cooled by immersion in a liquid or by thermalcoupling to a cold plate. Rack 1 houses a number of AC power units andcooling modules 2 and has expansion room for further modules inadditional equipment shelves 3. The modules are removable forreplacement or repair. FIG. 1 shows a typical packing density ofmodules. Only one shelf 3 of the rack 1 is filled with cooling units 2.The others could be similarly filled with cooling units 2. Cooling units2 are inserted from the front of the rack.

In FIG. 2, there is shown a simplified cross-sectional side view of theequipment rack shown in FIG. 1. The front 16, side 15 and rear 17 of therack 1 are shown.

The equipment rack 1 additionally houses towards the rear: a firstplenum chamber 18 (a pressure equalization device); a second plenumchamber 19; a pump 21; a header tank 20; a heat exchanger 22; a firstliquid connector 23; and a second liquid connector 24.

Liquid connections 11 and 12 are also shown on the cooling unit 2. Theseinterconnect with a system of pipes in a second liquid cooling stage ofthe system, which will be described further below. These are normally atthe rear 17 of the rack, although in circumstances where rear access isnot convenient, they could be at the front 16 of the rack. In thisexample, the cooling unit 2 has one liquid inlet and one liquid outlet,serving two independently cooled sealable modules 41 within each coolingunit 2.

The first plenum chamber 18 collects cooling liquid from a number ofcooling units 2. The second plenum chamber 19 distributes cooling liquidto a number of cooling units 2. The pump 21 assists circulation of thecooling liquid via the plenum chambers 18 and 19. The header tank 20 isfor the cooling liquid circulated by the pump 21. The heat exchanger 22transfers heat from liquid in the second liquid cooling stage to liquidin a third liquid cooling stage. Liquid connectors 23 and 24 carryliquid in the third cooling stage to and from the heat exchanger toequipment outside the rack.

Thus, a first stage of cooling electronic components has now beendescribed. The first stage provides a low-cost cooling module, usingnon-forced cooling (in this case using conduction and convection througha cooling liquid to transport heat) and the ability by some means todetach and replace any faulty module with a module that is workingcorrectly. There may be any number from one to a large number of suchmodules in a system.

In the first stage, at least one sealable module 41 is used. Eachsealable module 41 houses one or more electronic circuit boards, powersupply units, DC to DC power converters or disk drives to be cooled.Heat is removed from the heat-generating electronic components to thefirst cooling liquid 66 contained within the sealable module and is thentransmitted from the first cooling liquid 66 via the conduction surface71 to a second cooling liquid flowing through the cold plate base 22.

The second cooling liquid is used in a second stage of cooling and ameans of circulating the cooling liquid is provided so as to carry heataway from the first stage. A third stage of cooling can also be used toavoid the use of liquids flowing through cooling units under highpressure.

Further intermediate stages of heat transfer also use liquid to carryheat to a final heat exchanger. Further cooling stages desirably includecooling liquid flow-rate management for the different stages of thesystem, and pressure management, in order to avoid high cooling liquidpressure in sealable modules, whilst allowing liquid to be pumpedeffectively to a final heat sink. The system thereby uses multiplestages of heat transfer using liquids in all stages up to the final heatexchanger.

Sufficient heat is removed to keep components within their specifiedtemperature range but not significantly more than that. Additional heattransfer and lower temperatures that allow alternative operating modessuch as “over-clocking” of processors are possible using the system butnot necessary in normal operation, since additional energy is consumedin achieving these lower operating temperatures and the alternativemodes of operation are not in common use in large scale systems.

Referring now to FIG. 8, there is shown a schematic view of athree-stage cooling system comprising a single cooling unit 2. Thecooling unit 2 houses two sealable modules 41, each of which has a firstcooling stage 113 using liquid convection and a second cooling stage 114in which second cooling liquid is circulated outside the cooling unit 2.Liquid flow to the two sealable modules 41 is provided via flow splitter55 and liquid flow from the modules 41 is combined in flow combiner 57.The system further comprises: quick release connectors 111; pipes forsecond cooling liquid 112; first pump 116; pump control 117; header tank109; heat exchanger 118; second pump 119; pump control 120; pipes forthird cooling liquid 126; and heat exchanger 121.

Pipes 112 are joined via quick release devices 111 that also contain ameans of isolating the second cooling liquid in the cooling module andpipes. When the cooling unit 2 is connected, the liquid can flownormally but when the cooling unit 2 is disconnected, these close offthe liquid flow and prevent liquid loss from the module or pipes.

The second cooling liquid circulates through the cold plate (not shown)of the sealable module 41 to a heat transfer device 118 and is thenreturned to the cold plate via a header tank 109, which regulates theliquid pressure in the second cooling stage so as to be only moderatelyhigher than atmospheric pressure, and a first pump 116 supplied withelectrical power from pump control 117, which can be varied to alter thepumping rate according to the amount of heat generated in sealablemodules 41 either locally or by means of a control signal from anexternal device. In the illustrated embodiment, the two cooling circuitswithin the cooling unit are connected in parallel via splitter 55 andcombiner 57. The flow-rate in each of the parallel circuits can beseparately pre-adjusted within the splitter 55 and/or combiner 57 totake account of different amounts of heat generated in each arm of thecooling system. The direction of liquid flow is shown by arrows.

The heat transfer device 118 has two liquid flow circuits. The heatedsecond cooling liquid from the cooling module circulates through a firstcircuit. Cool liquid in the third stage circulates through a secondcircuit. Heat is transferred from liquid in the first circuit via aheat-conducting interface to liquid in the second circuit, which thenflows away through pipes 126 to a final heat exchanger 121. Thedirection of liquid flow is again shown by arrows.

Heat exchanger 121 comprises: heat exchanger cooling plate 122; fan 123;and power supply 124. This is a conventional device, commonly referredto as a “dry cooler”, that may use atmospheric air as the final heatsink medium, this being blown by fan 123 driven by electrical power 124across a finned cooling plate or equivalent means of heat transfer tocool the circulating third cooling liquid. The cooled liquid is thenreturned via a pump 119 driven by electrical power 120 to the heattransfer device 118.

The three stages of liquid heat transfer are desirable in situationswhere the final heat exchanger is located some distance from theequipment to be cooled, for example on the roof of a building. In thiscase, the pressure difference between liquid circulating through thefinal heat exchanger 121 and the intermediate heat transfer device 118may be large. The second stage of heat transfer can use a liquid with amuch lower pressure, so that the potential for liquid to leak within thefirst stage cooling modules and damage the electronic circuit boards isgreatly reduced.

Since the second and third cooling liquids are not in contact with theelectronic circuit board 75 (not shown), their characteristics are lessconstrained than those of the first cooling liquid 66. Water, which hasthe highest specific heat capacity of any common liquid and has very lowcost, can be used effectively. An additive to reduce corrosion andbacterial contamination may optionally be used.

The probability of leakage of the second cooling liquid is greatlyreduced by limiting the pressure in the second cooling stage. Headertank 109 provides regulation of the pressure in this stage. If anyleakage should occur, the fluid may be distinguished from liquid used inthe first cooling stage by addition of a small amount of dye. Since thesecond cooling liquid can be water, a range of low-cost non-toxic dyesis available for this purpose.

Referring now to FIG. 9, there is shown a block diagram showing a largerscale three-stage cooling system with multiple cooling units accordingto the invention. Three cooling units, each with two sealable modulesaccording to the invention are shown but, typically, many more units canbe accommodated in a system. This example of a larger scale coolingsystem, comprises: cooling units 130, 131, 132; pipes 129; plenumchamber 147 to combine liquid flow from cooling units; plenum chamber148 to distribute liquid flow to cooling units; pump 134 for secondcooling liquid; header tank 143; heat exchanger 135; pump 136 for thirdcooling liquid; final heat exchanger 137; final coolant entry 138 tofinal heat exchanger; and final coolant exit 139 from final heatexchanger.

A number of cooling units 130, 131, 132 are mounted in a housing or rackwith an arrangement such as that shown in FIG. 1. The liquid flowthrough the cooling units 130, 131 and 132 is connected in parallelfashion via plenum chambers 147 and 148. Each of the cooling units 130,131 and 132 typically contains two or more sealable modules 41 and canbe disconnected and removed from the rack separately, using quickrelease connectors such as those shown in FIG. 8, for replacement orrepair. The number of cooling units can be extended, as indicatedschematically via the additional inputs 141 and outputs 142 of theplenum chambers to a large number.

Plenum chambers 147 and 148 are advantageously insulated to improveefficiency by reducing local heat loss and reducing transfer of heatbetween the input plenum and the output plenum. A further improvement isto locate plenum units directly in line with the connectors of coolingunits 130, 131, 132, thus simplifying pipe arrangements and reducing thetotal number of liquid connectors in a system.

The second cooling liquid flow is divided amongst the cooling modules bya parallel arrangement of pipes 129 from plenum chamber 148, typicallywith one set of pipes per cooling unit, each serving two sealablemodules 41 mounted therein. The flow rate to each sealable module 41 canbe varied by use of restrictors and baffle plates in the plenum chamber154 By adjusting the flow rate to each cooling unit independently, amore efficient system is produced with a more uniform temperature of theheated second cooling liquid from the various cooling units.

Heated liquid from the cooling units is returned via pipes 129 to plenumchamber 147, where it is combined and delivered via pipes 129 to heatexchanger 135. Cooled liquid from heat exchanger 135 is passed to headertank 152, which regulates the pressure of the second cooling liquid.

Pump 134 drives the circulation of the second cooling liquid by drawingliquid from the header tank and pumping it to plenum chamber 148. Theliquid is then distributed back to the cooling units via pipes 129. Pump134 may be similar to pump 116 of FIG. 8, but is desirably of a largerscale to pump liquid to several cooling units 130, 131, 132 instead ofone. Arrows show the direction of liquid flow. Heat transfer device 135has the same function as the heat transfer device 118 of FIG. 8, exceptthat it is desirably of a larger scale to transfer heat from severalcooling units 130, 131, 132 instead of one.

The third stage of the system uses a third cooling liquid to transferheat from the heat exchange device 135 to a final heat exchanger 137.Pump 136 is used to circulate the liquid. Atmospheric air or coolgroundwater, used as the final heat sink medium, enters the heatexchanger at 138 and leaves at 139. In this case, and especially insystems that cool large arrays of servers, the entropy of the liquidcarrying the heat may be low enough to be used for other purposes,rather than be dumped into the environment. It may be used as a sourceof energy for heating a building for human occupation or the generationof useful amounts of electrical power. In other circumstances, where anunusually high atmospheric temperature would otherwise reduce thetemperature difference between the source and atmospheric final heatsink to too low a level, excess heat may be diverted (by diverting someof the final liquid) to refrigeration (“chiller”) units, or additionalenergy or cost might be expended in the final heat exchanger (such asthe use of adiabatic “dry coolers”, which spray water into the air toreduce the effective ambient temperature, wet bulb temperature).

FIG. 9 also shows signal outputs E1, E2, E3 from cooling units 130, 131and 132. Further signal outputs (not shown) may be provided byadditional cooling units so that the full set is E1, E2, E3 . . . En.Also shown are control inputs B and C to pump 134 and pump 135respectively, and control input D to final heat exchanger 137. These canbe used for monitoring and control purposes. This will be explained inmore detail below.

Referring now to FIG. 10, there is shown a monitoring and control system140 for use with the three-stage cooling system of FIG. 9. The systemcomprises: data inputs 146; and pump control outputs 145.

A monitoring and control system is used to monitor the temperatures ofthe electronic devices to be cooled and to adjust the flow rate of thesecond cooling liquid, third cooling liquid or both to provide optimumcooling. Sensors on each electrical circuit board measure thetemperature of the electronic components and convert this information toanalogue or digital signals. FIG. 9 shows signal outputs E1, E2, E3,which can be extended to En, where n is the total number of coolingunits that contain temperature-sensing devices, and where each signalcontains information about the temperature of the one or more sealablemodules in each cooling unit. These outputs are sent to the controlsystem 140.

The control system 140 computes the optimum flow rate of second coolingliquid, the optimum flow rate of the third cooling liquid and the rateat which the final heat exchanger should operate and produces inresponse a control message B, C, D. Control messages B and C are used toturn on or off or vary the pumping rate of pumps 134 and 136respectively. In addition, the control system determines whether or notthe final heat exchanger needs to adjust its cooling rate, for exampleby altering the flow level of the final heat sink liquid or air, usingcontrol signal D.

The overall thermal capacity of the system is large, so that short termrelatively large increases in ambient temperature (and thus thetemperature of the final heat sink) can be accommodated without the keycomponents exceeding their maximum temperature ratings. The ambienttemperature on the hottest days may rise to a level where not all of theheat generated by the components to be cooled can be removed. Theoperating temperature of components may rise but the high thermalcapacity means that the rise takes place slowly and maximum temperaturesare not exceeded. During the cooler part of the diurnal cycle, more heatcan be removed than is generated. In this way the system can be operatedat locations with climates that would not otherwise allowrefrigeration-free cooling. The control system can be optimised to useambient temperature data, measured from external sensors and fromhistorical trend data and statistics, to ensure that the flow rates areoptimised. In the rare event that very exceptional temperatures occur,timely warning can be given to either reduce the effective temperatureof the final heat sink (such as by switching to an active mode ofcooling) or otherwise enable the system operator to take appropriateremedial actions.

In systems where “run-time hardware abstraction” of processing systemsis used (such as with “virtualization” or “run-time middleware”), themonitoring and control system is particularly important. In systems withhardware abstraction, the multiple electronic circuits boards(“hardware”) and multiple computer operating systems are not in one toone correspondence. When one circuit board is under high processingload, some activity can be shared with other boards. Processing isdistributed across the items of hardware. As a result, heat generated indifferent parts of the systems varies from time to time. Cooling ratesin different parts of the system may then be adjusted dynamically toalign with the changes in amounts of heat generated.

If the first cooling liquid in the sealable module 41 (and thus, thecircuit board 75) is to operate at a desired temperature, T_(case), andthe final heat sink is at a known temperature T_(hs), this defines thetemperature difference that the cooling system desirably provides, inthat ΔT=T_(case)−T_(hs). Since T_(case) is desirably restricted to nohigher than the maximum operating temperature of the circuit board,T_(case,max), then ΔT≦T_(case,max)−T_(hs).

Semiconductor manufacturers are increasingly reducing the maximumoperating temperature of their processors. This reduces the temperaturedifference, particularly when refrigeration is not employed to increasethe local temperature difference. A further difficulty arises when thefinal heat sink is at atmospheric temperature, which may be as high as40 degrees centigrade.

Reducing the thermal resistances in the system can assist to achieve thedesired temperature difference. Systems which transfer heat throughfluid flow may result in reduced thermal resistance compared withsystems which transfer heat, either in full or in significant part,through static, thermally-coupled bodies or through gases. For example,the flow rate of the second cooling liquid can be adjusted to reduce thethermal resistance between the sealable module 41 and the heat exchanger118. Additionally or alternatively, the area of contact between thesecond cooling liquid flowing through the channels 61 and the conductionsurface 71 can be varied to affect the thermal resistance.

The use of tightly-packed channels 61 increases the pressure drop (i.e.hydraulic pressure losses) in the cold plate which increases the pumpingcosts for the secondary cooling liquid circuit. The width of the channelcan be modified to reduce pressure losses and decrease the effect of theconduction surface 71 in the cold plate 22 in transferring heat intowater channels.

At one extreme, the channels 61 could be as wide or wider than thedimensions of the housing 81 to present a “flood plain”, rather than a“serpentine river”, of second cooling liquid. However, controlling flowof second cooling liquid in such an embodiment may be difficult andfeatures such as eddy currents may cause local build up of heat, whichis undesirable as it will reduce rate of heat transfer for adjoiningareas of conduction surface 71.

Therefore, the channel 61 width can be less than, but significant incomparison with a dimension of the housing 81. Optimisation of thecross-section of the channels 61 can improve the temperature difference.Channels 61 that are approximately 20% of the length of the longestchannel 61 may be defined by baffles that direct flow over specificareas. Also, the flow of the second cooling liquid can be portioned withthe channels 61 into sections such that the water is distributed intozones and slowed over areas with greatest heat flux (e.g. processors),the reintroduction of heat back into the primary coolant can beminimised, whilst the entropy of the extracted heat can be minimised.

Whilst a preferred embodiment and operating modes have been describedabove, the skilled person will appreciate that various modifications canbe made.

For example, cold plate 5 shown in FIG. 1 is optional and the skilledperson will appreciate that this need not be included. Instead ofinserting cooling units 2 into the equipment rack 1 from the front, theycould be inserted from the rear of the rack. Advantageously, the coolingunits could be adapted to carry only one sealable module in each unit,so that units may then be inserted both at the front and rear of therack. Electrical and liquid connections are then made in the middle ofthe rack. A single module unit is much lighter in weight than a twomodule unit and can be safely carried, fitted or removed by a singleperson.

Although intermediate stages of heat transfer and a final heat sink mayuse liquid cooling, other cooling mechanisms can alternatively be used,such as convection cooling.

The skilled person will also recognise that this can be implemented andoperated in a number of different ways. Referring now to FIG. 11A andFIG. 11B, there are shown side views of a second embodiment of asealable module 150, which is an alternative to that described above.The sealable module 150 comprises an outer cover 151 and housing 152.Although the second embodiment differs slightly from the firstembodiment, the skilled person will appreciate that many features of thetwo embodiments are interchangeable.

The channels in the cold plate 60 are closed by the outer cover 151. Theassembly, which defines the channels, can be termed a “water jacket”,because it dictates the flow of water (as second cooling liquid) forheat to flow over a heat transfer surface, such as a conduction surface71.

Referring now to FIG. 11C, there is shown a more detailed view of thesealable module according to FIG. 11A and FIG. 11B. The sealable module150 also comprises a heat transfer surface 153, which together with theouter housing 151 defines channels 156. Also, the heat transfer surface153 and lid 152 (which can be termed a base or housing) define an innerchamber with an internal volume for locating the electronic device 155to be cooled and the first cooling liquid (not shown). Connectors 154are also shown, for allowing flow of the second cooling liquid throughchannels 156.

This alternative embodiment of a module may therefore comprise an innerhousing, containing the first cooling liquid and one or moremotherboards, substantially sealed (except for a filling port). A heattransfer surface 153 has a gasketed interface with the inner housing andfins facing into the first cooling liquid, shaped to match the profileof the at least one motherboard. The outer housing 151 also has agasketed interface with the heat transfer surface 153, and contains atleast one secondary cooling liquid circuit channel 156, formed withbaffles on the heat transfer surface 153 or on the interior surface ofthe outer housing 151. The at least one channel 156 is optimised todirect the second cooling liquid appropriately over the heat transfersurface 153 with minimised pressure loss. The outer housing 151 hasquick-connect hydraulic connectors 154 to enable the channel 156 to beconnected to the inlet and the outlet of a rack second cooling liquidsupply.

This design allows much tighter integration between the inner housing,heat transfer surface 153 and outer housing 151 potentially to make asmaller unit, which may allow increased packing density of modules. Finsor baffles may be provided on both sides of heat transfer surface 153.These can, for example, provide additional flow control of secondcooling liquid and increase surface area for conduction.

The materials for the present invention can be varied. Metal materialsare good conductors, but expensive. Also, if the heat transfer surface153 or conduction surface 71 (which can be similar or identical) aremade of metal and are large, they may be more likely to warp, puttingstress on seals, especially if multiple sealable modules are mountedthereon, meaning areas of potentially different temperature.

In contrast, synthetic plastic materials are inferior conductors incomparison with metal. Known thermally conductive plastics typicallyconduct 20 W/mK compared to 141 W/mK for aluminium. Higher performancethermally conductive plastics are also usually electrically conductive,which is not a desirable characteristic. However, these materials areless expensive, of lighter weight and are less likely to corrode in thepresence of hot water (notwithstanding the possible addition ofcorrosion inhibitors to the second cooling liquid) than metals.

Optimisation of the temperature difference by control of second coolingliquid flow rates, area of heat transfer and channel cross-section canallow the use of plastic without significant reduction in thetemperature difference. In particular, plastic can be used for the basepart 22 of the cold plate or the outer housing (“water jacket”)described above. A plastic material reduces the amount of heattransferred through the outer wall of the base part 22 or outer housing151 and into the local ambient environment, reducing heat lost in thisway and increasing efficiency of heat removal into second coolingliquid.

Additionally or alternatively, the conduction surface 71 or heattransfer surface 153 can be made of plastic. This may provide additionalexpansion capacity for the first cooling liquid within the sealedhousing. Such a material might be co-moulded with a rigid centralthermal conductive plastic and peripheral ring of flexiblenon-conductive plastic.

A number of features of the embodiment described above will beunderstood as optional to the skilled person and might be omitted. Thesemay include insulation 73, which could additionally be made of a fireretardant material and quick release connectors 111. Also, the skilledperson will understand that alternative constructions for the cold plate60 or outer housing 151 can be used and that the projections 96, 97 canbe of different length and cross-section to that described.

Although an embodiment described above uses a cooling unit 2 in whichone sealable module 41 is affixed to a cold plate 60, it will berecognised that two or more sealable modules 41 might be coupled to acommon cold plate. Also, cooling units 2 may be inserted from both backand front of the rack.

The electronic circuit board can be combined with housing 81 to form anintegrated assembly. This provides a means of connection to electroniccomponents directly from the circuit board, thus reducing the risk ofliquid leaking from cable seals in the module, reducing the overallwidth of a sealable module and increasing packing density of coolingunits within a rack.

The data transmission cables 46 can be replaced by fibre optic cables,optical or infra-red ports or wireless connection between the electroniccircuit board and the exterior of the sealable module. Power supplyconnections would normally be wired, even when the data transmission isby other means, although alternative power supplies may be employed,whilst avoiding fluid leakage.

The second cooling liquid may be distributed via pipes with individualflow control valves, rather than a plenum chamber. The valves may belocally adjusted or controlled automatically by a central monitoring andcontrol system similar to control system 140 shown in FIG. 10 inresponse to temperature and status information from the componentshoused within the sealable modules 41. Also, in the system of FIG. 8,heat exchanger 121 can alternatively be replaced by cool groundwater ora number of configurations including bypass circuits to transfer someheat to refrigeration systems. To provide resilience and redundancyadditional “back-up” pumps and circuits may optionally be provided.

In particular, a cooling system with redundancy can use a heat exchangerarrangement that receives the liquid coolant, transfers heat from afirst portion of the liquid coolant to a first heat sink and transfersheat from a second portion of the liquid coolant to a second heat sink.However, the first and second heat sinks are isolated from one another.

Referring to FIG. 12, there is depicted a schematic diagram of a coolingsystem with cabinet-level redundancy. It should be noted that thisdiagram shows a simplistic and schematic understanding of the system, asadditional components may be required, some of which are identifiedbelow.

The cooling system is housed within a cabinet 250 and may comprise: afirst module 251; a second module 252; a third module 253; a pipingsystem 260; a first heat exchanger 281; a second heat exchanger 282; afirst coolant pump 291; and a second coolant pump. Although threemodules 251, 252, 253, two heat exchangers 281, 282 and two coolantpumps 291, 292 are shown, this is for illustration purposes only and itwill be understood that fewer (down to even one) or more modules may beused, more than two heat exchangers and fewer or more than two coolantpumps are possible. Each of the modules may be in accordance withsealable modules described above. Although reference here is made tocabinet-level redundancy, it may be understood that not all of thecomponents need be contained within a single cabinet, although it ispreferable that at least the multiple modules are housed within onecabinet.

The piping system 260 may be used for receiving warmed liquid coolantfrom the modules. The first module 251 has a liquid output port 271, thesecond module 252 has a liquid output port 272 and the third module 253has a liquid output port 273. In this embodiment, the output liquidcoolant is combined (a plenum chamber, not shown, can be used for thispurpose, as discussed above), but this is not necessary. The pipingsystem 260 separates the combined liquid coolant into two portions, thefirst portion of the coolant being carried by a first output pipe 274 tothe first heat exchanger 281 and the second portion of the coolant beingcarried by a second output pipe 275 to the second heat exchanger 282.

The first and second portions of the liquid coolant are therebytransferred to a heat exchanger arrangement made up from the first heatexchanger 281 and the second heat exchanger 282. Each of the two heatexchanges 281, 282 transfers the heat received to a respective heat sink(not shown). These heat sinks may comprise further coolants, which maybe fluid and possibility liquid, and separate fluid coolants canoptionally be combined downstream. A subsequent stage heat exchangersystem can optionally be provided to receive the multiple heat sinkcoolants and to transfer heat from these coolants to a common outputheat sink. Redundancy in the pumping of the sink coolant or coolants canbe effected, for example by providing two or more pumps in series or inparallel. It will be appreciated that other forms of heat exchangerarrangement may be possible which transfer two separated portions of theliquid coolant to two separate heat sinks, isolated from one another.

A similar configuration is shown for the transfer of the cooled coolantfrom the two heat exchangers back to the sealable modules via the pipingsystem 260. A first portion of the coolant is carried by a first inputpipe 264 from the first heat exchanger 281 and a second portion of thecoolant is carried by a second output pipe 265 from the second heatexchanger 282. The cooled coolant is then transferred back to themodules. The first module 251 has a liquid input port 261, the secondmodule 252 has a liquid input port 262 and the third module 253 has aliquid output port 263.

This use of multiple heat exchangers for the warmed coolant that isoutput by one or more modules allows redundancy, for load sharing andparticularly to cope with possible failure of one or more component,especially a part of the piping system 260 or one of the heatexchangers. In the event that one heat exchanger is unable to transferheat to its heat sink, another heat exchanger is configured to transferat least some (or all) of the heat that would have been transferred awayto another heat sink.

The heat exchangers can be configured to transfer an approximately equalproportion of the heat received from the modules to the first heat sinkas the second heat sink. In other words, the heat exchanger arrangementmay divide the heat output between the two heat sinks roughly equally,such that the two share the load. However, the ratio of heat transfermay be controlled or may be set differently (for example, 45%:65%,40%:60%, 35%:75%, 30%:70%, 25%:75%, 20%:80%, 10%:90%, 5%:95%). Thecapacity of the first heat sink and the second heat sink may be the samein many embodiments.

Redundancy can additionally (or alternatively) be provided in respect ofpumps, by use of independent pumps. In FIG. 12, the first pump 291 isplaced in the first output pipe 274 and the second pump 292 is placed inthe second output pipe 275, but other arrangements are possible and thepumps need not be placed in separated portions of the piping system 260.When the final heat sink 121 is close to the equipment to be cooled andthe pressure of the cooling liquid circulating in this stage can belower, a second stage of cooling might be omitted. In this case, thesecond cooling liquid circulates directly through the final heatexchanger. Pump 116 and intermediate heat transfer device 118 or 135 areomitted. As with the embodiment illustrated in FIG. 8, the three-stagesystem shown in FIG. 9 could be reduced to two stages if the liquidpressure in the final stage were low enough.

Further improvements to the cooling system configuration described abovefor more complex cooling systems are also contemplated. In particular,data processing or computer server centres use electronic componentswith increasing greater heat output, requiring more cooling power thanbefore. At the same time, redundancy is added to the cooling system toavoid failures of specific components or processes causing a breakdownof a portion or the whole of the system. Adding such redundancy increasecosts and can reduce overall efficiency.

Referring to FIG. 13, there is shown a cooling apparatus 160, comprisinga housing 161; a heat generating electronic component 162; a first heatexchanger 163; and a second heat exchanger 164. The housing 161(optionally in conjunction with the first heat exchanger 163, secondheat exchanger 164 or both) defines a volume 165, in which the heatgenerating electronic component 162 is located.

Prior to operation, the volume 165 is filled with a liquid coolant.WO-2010/130993 and US-2010/0290190 describe techniques for filling avolume such as volume 165 with liquid coolant, in particular accountingfor any pressure changes during operation. The sealable module 160 issealed such that the liquid coolant should not escape.

During operation, the heat generating electronic component 162 heats theliquid coolant in the volume 165 and heat is transferred from the liquidcoolant to heat sinks exterior the sealable module 160 via the firstheat exchanger 163 and the second heat exchanger 164. The respectiveheat sinks for the first and second heat exchangers are isolated fromanother. In this context, the term heat sink is used simply as ashorthand for the device or arrangement through which heat istransferred away from the sealable module 160.

Implementation details of one embodiment in accordance with FIG. 13 arenow discussed. In that context and with reference to FIG. 14, there isshown a cross-sectional exploded view of an embodiment of a sealablemodule comprising a heat generating electronic component in accordancewith FIG. 13. This embodiment is largely based on the design shown inFIG. 5. Where the same components or parts as shown in FIG. 5 are used,identical reference numerals have been employed.

The sealable module 170 comprises a heat generating electronic component185. It further comprises: a first finned conduction surface 71 formingpart of a first cold plate 60; first liquid flow channels 61 adjacentthe conduction surface 71; a second finned conduction surface 186forming part of a second cold plate 180; and second liquid flow channels181 adjacent the second conduction surface 186. Fixings are omitted fromthis drawing, for the sake of clarity.

As per FIG. 5, the heat generating electronic component 185 comprises: acircuit board 75; a small electronic component 68; a large electroniccomponent 76; and a rear-mounted electronic component 184.

The sealable module 170 further comprises: a first sealing gasket 64; asecond sealing gasket 182; a third sealing gasket 188; pin-finprojections 65 on the first conduction surface 71; pin-fin projections183 on the second conduction surface 186; a first cover plate 78 for theside of the first cold plate 60 opposite to the fins on the firstconduction surface 71; and a second cover plate 187 for the side of thesecond plate 170 opposite to the fins on the second conduction surface186.

Each of the first cold plates 60 and second cold plates 180 arefabricated with two faces, each with a separate function. The firstconduction surface 71 and the second conduction surface 186 are apin-finned plate, forming one face of the respective cold plate. The twocold plates may be attached together, as shown in the drawing, in orderto create a sealed volume in which the heat generating electroniccomponent 185 may be housed. The sealing gaskets ensure that theassembled capsule is substantially sealed against liquid loss or ingressof air. Mounting fixtures (not shown) are provided for the heatgenerating electronic component 185. The fins 65 of the first conductionsurface 71 and fins 183 of the second conduction surface 186 face thecircuit board 75. In some cases, components of significant size may bepresent on both sides of the board, as shown. Alternatively (but not asshown), components of significant size may only be present on one sideof the board. A small gap is provided between the ends of the fins 65and the ends of fins 183 and the components. The fins have an elongatedcross-section and their height varied, so as to maintain a small gapbetween the variously sized components on the electronic circuit boardand the tops of the fins. This is shown for both fins 65 and fins 183 inFIG. 14.

For the first cold plate 60, the first cover plate 78 allows firstchannels 61 to be defined. These allow liquid flow for a second liquidcoolant (distinct from the liquid coolant within the volume defined bythe sealed module), which is used for heat transfer away from thesealable module 170. Similarly, the second cover plate 187 createssecond channels 181 for a third liquid coolant to flow and therebyconvey heat away from the sealable module 170 through an independentpath. The second and third liquid coolants are independently controlled,so as to provide two isolated heat sinks for the sealable module 170.

Further design and implementation features of this embodiment may beadded or adjusted in accordance with the details disclosed previouslypublished WO-2010/130993 and US-2010/0290190.

Although an embodiment has been described, the skilled person willunderstand that various variations and modifications may be made. Forexample, although the embodiment shown in FIG. 14 uses two separate coldplates, other types of heat exchangers can be employed. Similarly,whilst there will be understandable advantages from the constructionshown in FIG. 14, in which the two heat exchangers are provided onopposite walls of the sealable module, it will be understood that theheat exchangers may alternatively be implemented in the same wall, oradjacent walls. An embodiment along these lines is described below.Equivalently, it will be understood that each heat exchanger need nottake up the whole of a wall of the sealable module, but could form partof a wall. In fact, more than two heat exchangers might be provided insome embodiments.

With reference to FIG. 14, the first conduction surface 71, secondconduction surface 186 or both need not be provided with fins 65 or fins183, and alternatives that would be well known to the skilled person arealso possible. Moreover, the fins on one or both conduction surfacesneed not have shapes, sizes or both that are adapted in accordance witha shape of the electronic component.

Although a housing that is separate from the first cold plate 60 andsecond plate 180 is not shown in FIG. 14, it will be understood thatthis is possible as well as various other structural configurationsinvolving cold plates and integrated or distinct housings. Whilst theheat sink for the first heat exchanger and heat sink for the second heatexchanger take the form of liquid coolants in the embodiment in shown inFIG. 14, it will be appreciated that other kinds of heat sinks may beused.

Further alternative embodiments are now briefly described at a schematiclevel. The skilled person will appreciate that further implementationdetails of such embodiments may be similar to the embodiment describedabove or similar to other known arrangements.

Referring to FIG. 15, there is shown a schematic diagram of analternative cooling apparatus. Where the same features are shown as inother drawings, the same reference numerals have been used. The coolingapparatus 200 comprises: a housing 201; a heat generating electroniccomponent 162; a first heat exchanger 210; and a second heat exchanger220. The volume defined by the housing 201, a first heat exchanger 210and a second heat exchanger 220 is filled with a liquid coolant 202. Thefirst heat exchanger 210 has a coolant input 211 and a coolant output212 and the second heat exchanger 220 has a coolant input 221 and acoolant output 222, separate from those of the first heat exchanger 210.

This cooling apparatus 200 operates in a similar fashion to the coolingapparatus 160 shown in FIG. 13. During operation, the heat generatingelectronic component 162 heats the liquid coolant in the volume 202 andheat is transferred from the liquid coolant to heat sinks exterior thesealable module 200 via the first heat exchanger 210 and the second heatexchanger 220. The respective heat sinks for the first and second heatexchangers are isolated from another. The first heat exchanger 210 andthe second heat exchanger 220 are arranged on the same side of the heatgenerating electronic component 162 and make up part of the same wall ofthe volume that they define together with the housing.

Referring next to FIG. 16, there is shown a schematic diagram of afurther alternative cooling apparatus. Again, the same features as shownin other drawings are indicated by identical reference numerals. Thecooling apparatus 300 comprises: a heat generating electronic component162; a first heat exchanger 310; a second heat exchanger 320; a housing301; and a wicking material 305. The housing 301 and the wickingmaterial 305 together define a vapour chamber attached to the heatgenerating electronic component 162.

The first heat exchanger 310 has a coolant input 311 and a coolantoutput 312 and the second heat exchanger 320 has a coolant input 321 anda coolant output 322, separate from those of the first heat exchanger310. Thus, the first heat exchanger 310 and the second heat exchanger320 (which are typically separate cold plates) act in parallel and eachcan provide redundancy should the other fail.

Referring now to FIG. 17, there is shown a schematic diagram of a yetfurther alternative cooling apparatus. Once more, where the samefeatures are shown as in other drawings, these are indicated byidentical reference numerals. The cooling apparatus 400 comprises: aheat generating electronic component 162; a conductive fixing material405; and a heat exchanger 410. The heat generating electronic component162 has raised parts (for example, where the heat generating electroniccomponent 162 is a circuit board, it may have components mounted uponit) comprising: a first component 106 and a second component 107. Theconductive fixing material 405 is a bonding material which also acts asa thermal interface to conduct heat from the heat generating electroniccomponent 162 to the heat exchanger 410.

The heat exchanger 410 differs from previously described heatexchangers. It is a cold plate with two separate coolant inlets: firstinlet 411 and second inlet 412. It also has two separate coolantoutlets: first outlet 421 and second outlet 422. Within heat exchanger410, there are two separate, isolate channels through which two separateflows of coolant are defined: a first flow from first inlet 411 to firstoutlet 421; and a second flow from second inlet 412 to second outlet422. These two flows are isolated from one another and therefore act astwo parallel heat sinks for the heat exchanger 410. This providesredundancy, such that failure of one flow can be compensated by theother. In order for the transfer of heat from the heat generatingelectronic component 162 to the two coolant channels to be balanced (andavoid placing a much greater load on one channel than the other), thetwo channels may be provided to cross over one another in threedimensions, such as using a helical (or spiral) form. The two channelsmay thereby cover the entire surface of the heat exchanger 410 adjacentthe heat generating electronic component 162 (referred to as theconduction surface above) Without such adaptations, a situation mayarise if one channel fails that the other channel may not havesufficient thermal capacity to cope with the heat transfer requiredacross the whole heat transfer surface.

Although this embodiment has been described with a conductive thermalinterface between the heat generating electronic component 162 and theheat exchanger 410, it will be recognised that it can equally beimplemented with a convective thermal interface. This might beimplemented by use of a coolant-filled volume, as described withreference to FIGS. 13 to 15. Also, it will be understood that individualfeatures from the different embodiments can also be combined asappropriate to gain advantage from the specific benefits discussedabove.

1. A cooling apparatus configured to use a liquid coolant for carryingheat removed from an electronic device, the cooling apparatuscomprising: a heat exchanger arrangement, configured to receive theliquid coolant, to transfer heat from a first portion of the liquidcoolant to a first heat sink and to transfer heat from a second portionof the liquid coolant to a second heat sink; and wherein the first andsecond heat sinks are isolated from one another.
 2. The coolingapparatus of claim 1, wherein the heat exchanger arrangement isconfigured to transfer at least some of the heat from the first portionof the liquid coolant to the second heat sink, in the event that theheat exchanger arrangement is unable to transfer heat from the firstportion of the liquid coolant to the first heat sink.
 3. The coolingapparatus of claim 1, further comprising: a sealable module that definesa volume in which the electronic device and the liquid coolant can belocated, such that the liquid coolant can remove heat generated by theelectronic device; and wherein the heat exchanger arrangement comprises:a first heat exchanger, arranged to receive the first portion of theliquid coolant at a first part of the volume and to transfer heat fromthe first portion of the liquid coolant to the first heat sink; and asecond heat exchanger, arranged to receive the second portion of theliquid coolant at a second part of the volume and to transfer heat fromthe second portion of the liquid coolant to the second heat sink.
 4. Thecooling apparatus of claim 3, wherein the sealable module comprises ahousing, the first heat exchanger comprising a first conduction surfacethat cooperates with the housing so as to define at least part of thevolume.
 5. The cooling apparatus of claim 4, wherein the first heatexchanger further defines a first channel for receiving a first furtherliquid coolant, the first conduction surface separating the volume andthe first channel to allow conduction of heat between the volume and thefirst channel through the first conduction surface.
 6. The coolingapparatus of claim 4, further comprising the electronic device andwherein at least a portion of the first conduction surface or housing isshaped in conformity with the shape of the at least one electronicdevice.
 7. The cooling apparatus of claim 4, wherein the second heatexchanger comprises a second conduction surface that cooperates with thehousing so as to define at least part of the volume.
 8. The coolingapparatus of claim 7, wherein the second heat exchanger further definesa second channel for receiving a second further liquid coolant, thesecond conduction surface separating the volume and the second channelto allow conduction of heat between the volume and the second channelthrough the second conduction surface.
 9. The cooling apparatus of claim7, further comprising the electronic device and wherein at least aportion of the second conduction surface or housing is shaped inconformity with the shape of the electronic device.
 10. The coolingapparatus of claim 7, wherein the housing and the first and secondconduction surfaces define the volume.
 11. The cooling apparatus ofclaim 10, further comprising the electronic device having an axis ofelongation and wherein the first and second conduction surfaces eachhave respective axes of elongation that are substantially parallel tothe axis of elongation of the electronic device.
 12. The coolingapparatus of claim 1, wherein the electronic device comprises a circuitboard, defining a substantially planar form.
 13. The cooling apparatusof claim 1, further comprising: a piping system for receiving the liquidcoolant, the piping system being arranged to separate the first andsecond portions of the liquid coolant and to transfer the first andsecond portions of the liquid coolant to the heat exchanger arrangement.14. The cooling apparatus of claim 13, wherein the heat exchangerarrangement comprises: a first heat exchanger, arranged to receive thefirst portion of the liquid coolant and to transfer heat from the firstportion of the liquid coolant to the first heat sink; and a second heatexchanger, arranged to receive the second portion of the liquid coolantand to transfer heat from the second portion of the liquid coolant tothe second heat sink.
 15. The cooling apparatus of claim 14, furthercomprising: a first coolant pump, arranged to pump the liquid coolant inthe piping system; and a second coolant pump, arranged to pump thesecond sink coolant, independently from the first coolant pump.
 16. Thecooling apparatus of claim 14, wherein the first heat sink comprises afirst sink coolant and wherein the second heat sink comprises a secondsink coolant, the first and second sink coolants being isolated from oneanother.
 17. The cooling apparatus of claim 16, wherein the first andsecond sink coolants are liquids.
 18. The cooling apparatus of claim 16,further comprising: a second heat exchanging arrangement, configured toreceive the first and second sink coolants and to transfer heat from thefirst and second sink coolants to a common output heat sink.
 19. Thecooling apparatus of claim 16, further comprising: a first outer pump,arranged to pump the first sink coolant; and a second outer pump,arranged to pump the second sink coolant, independently from the firstouter pump.
 20. The cooling apparatus of claim 13, further comprising: aprimary stage heat exchanger arrangement, configured to transfer heatgenerated by the electronic device to the liquid coolant; and whereinthe piping system is arranged to receive the liquid coolant from theprimary stage heat exchanger arrangement.
 21. The cooling apparatus ofclaim 20, further comprising: a plurality of sealable modules, eachdefining an internal volume in which a respective electronic device islocated, each sealable module further comprising a respective primarystage heat exchanger configured to transfer heat generated by therespective electronic device to the liquid coolant, such that theprimary stage heat exchanger arrangement comprises the respectiveprimary stage heat exchanger arrangement of each of the plurality ofsealable modules.
 22. The cooling apparatus of claim 21, wherein theplurality of sealable modules are housed in a cabinet.
 23. The coolingapparatus of claim 22, wherein the heat exchanger arrangement is alsohoused in the cabinet.
 24. A method of operating a cooling system thatuses a liquid coolant for carrying heat removed from an electronicdevice, comprising: receiving a first portion of the liquid coolant at aheat exchanger arrangement, so as to transfer heat from the firstportion of the liquid coolant to a first heat sink; and receiving asecond, separate portion of the liquid coolant at the heat exchangerarrangement, so as to transfer heat from the second portion of theliquid coolant to a second heat sink; and wherein the first and secondheat sinks are isolated from one another.
 25. The method of claim 24,further comprising: transferring at least some of the heat from thefirst portion of the liquid coolant to the second heat sink at the heatexchanger arrangement, in the event that the heat exchanger arrangementis unable to transfer heat from the first portion of the liquid coolantto the first heat sink.
 26. The method of claim 24, further comprising:operating the electronic device in a volume defined by a sealablemodule, the liquid coolant also being located within the volume, suchthat the liquid coolant can remove heat generated by the electronicdevice; wherein the heat exchanger arrangement comprises: a first heatexchanger; and a second heat exchanger; and wherein the first portion ofthe liquid coolant is received by the first heat exchanger at a firstpart of the volume and the second portion of the liquid coolant isreceived by the second heat exchanger at a second part of the volume.27. The method of claim 24, wherein the heat exchanger arrangementcomprises: a first heat exchanger; and a second heat exchanger, themethod further comprising: receiving the liquid coolant in a pipingsystem; separating the first and second portions of the liquid coolantin the piping system; transferring the first and second portions of theliquid coolant to the first and second heat exchangers respectivelyusing the piping system.
 28. The method of claim 27, further comprising:transferring heat generated by the electronic device to the liquidcoolant using a primary stage heat exchanger arrangement; and whereinthe step of receiving the liquid coolant in the piping system comprisesreceiving the liquid coolant from the primary stage heat exchangerarrangement.
 29. The method of claim 28, further comprising: operatingeach of a plurality of electronic devices within an internal volume of arespective sealable module; transferring heat generated by eachelectronic device to the liquid coolant using a respective primary stageheat exchanger that forms part of the respective sealable module, suchthat the primary stage heat exchanger arrangement comprises therespective primary stage heat exchanger of each of the plurality ofsealable modules.
 30. The method of claim 24: wherein the step ofreceiving the first portion of the liquid coolant at the heat exchangerarrangement comprises transferring heat received from the first portionof the liquid coolant to a first sink coolant; and wherein the step ofreceiving the second portion of the liquid coolant at the heat exchangerarrangement comprises transferring heat received from the second portionof the liquid coolant to a second sink coolant, the first and secondsink coolants being isolated from one another.
 31. The method of claim30, wherein the first and second sink coolants are liquids.
 32. Themethod of claim 28, further comprising: receiving the first and secondsink coolants at a heat exchanging arrangement; and transferring heatfrom the first and second sink coolants to a common output heat sinkusing the heat exchanging arrangement.
 33. A method of manufacturing acooling system that uses a liquid coolant for carrying heat removed froman electronic device, comprising: providing a first heat exchanger forreceiving a first portion of the liquid coolant, so as to transfer heatfrom the first portion of the liquid coolant to a first heat sink; andproviding a second heat exchanger in parallel with the first heatexchanger for receiving a second, separate portion of the liquidcoolant, so as to transfer heat from the second portion of the liquidcoolant to a second heat sink; and wherein the first and second heatsinks are isolated from one another.
 34. A cooling apparatus comprising:an interfacing component configured to receive an electronic device anddefining a thermal interface through which heat generated by theelectronic device can be transferred away from it; and a heat exchangerarrangement, coupled to the thermal interface and configured to transferheat generated by the electronic device via the thermal interface to afirst heat sink and a second heat sink; and wherein the first and secondheat sinks are isolated from one another.
 35. The cooling apparatus ofclaim 34, wherein in the event that a part of the heat exchangerarrangement is unable to transfer heat to the first heat sink, the heatexchanger arrangement is configured to transfer at least some of theheat that would have been transferred to the first heat sink to thesecond heat sink.
 36. The cooling apparatus of claim 34, wherein theinterfacing component comprises a housing that defines at least part ofa volume in which a coolant and at least part of the electronic devicecan be located, such that the coolant can provide the thermal interfaceby convection and wherein the heat exchanger arrangement comprises: afirst heat exchanger, having a part located in a first part of thevolume and configured to transfer heat from a first portion of thecoolant to the first heat sink; and a second heat exchanger, having apart located in a second part of the volume and arranged in parallelwith the first heat exchanger, the second heat exchanger beingconfigured to transfer heat from a second portion of the coolant to thesecond heat sink.
 37. The cooling apparatus of claim 36, wherein thefirst heat exchanger comprises a first conduction surface thatcooperates with the housing so as to define at least part of the volume.38. The cooling apparatus of claim 37, wherein the first heat exchangerfurther defines a first channel for receiving a first outer liquidcoolant, the first conduction surface separating the volume and thefirst channel to allow conduction of heat between the volume and thefirst channel through the first conduction surface.
 39. The coolingapparatus of claim 37, further comprising the electronic device andwherein at least a portion of the first conduction surface or housing isshaped in conformity with a shape of the electronic device.
 40. Thecooling apparatus of claim 37, wherein the second heat exchangercomprises a second conduction surface that cooperates with the housingso as to define at least part of the volume.
 41. The cooling apparatusof claim 40, wherein the second heat exchanger further defines a secondchannel for receiving a second outer liquid coolant, the secondconduction surface separating the volume and the second channel to allowconduction of heat between the volume and the second channel through thesecond conduction surface.
 42. The cooling apparatus of claim 40,further comprising the electronic device and wherein at least a portionof the second conduction surface or housing is shaped in conformity witha shape of the electronic device.
 43. The cooling apparatus of claim 40,wherein the housing and the first and second conduction surfaces definethe volume.
 44. The cooling apparatus of claim 43, wherein the housing,the first and second conduction surfaces and the electronic devicedefine the volume.
 45. The cooling apparatus of claim 44, wherein thevolume further comprises a wicking material such that a vapour chamberis formed.
 46. The cooling apparatus of claim 43, further comprising theelectronic device having an axis of elongation and wherein the first andsecond conduction surfaces each have respective axes of elongation thatare substantially parallel to the axis of elongation of the electronicdevice.
 47. The cooling apparatus of claim 46, wherein the electronicdevice and the first and second conduction surfaces each have asubstantially planar form and wherein the planes of the first and secondconduction surfaces are substantially parallel to the plane of theelectronic device.
 48. The cooling apparatus of claim 34, wherein theelectronic device comprises a circuit board, defining a substantiallyplanar form.
 49. The cooling apparatus of claim 46, wherein the firstand second conduction surfaces are located on the same side of theelectronic device.
 50. The cooling apparatus of claim 46, wherein thefirst and second conduction surfaces are located on opposite sides ofthe electronic device.
 51. The cooling apparatus of claim 36, whereinthe coolant comprises a liquid.
 52. The cooling apparatus of claim 36,further comprising the coolant.
 53. The cooling apparatus of claim 34,wherein the heat exchanger arrangement or the interfacing componentcomprises a conduction surface and wherein the heat exchangerarrangement further comprises: a first piping system, arranged to carrya first coolant, to receive heat directly from the conduction surfaceand to transfer heat through the conduction surface to the firstcoolant; and a second piping system, arranged in parallel with the firstpiping system to carry a second coolant, to receive heat directly fromthe conduction surface and to transfer heat through the conductionsurface to the second coolant; and wherein the first and second coolantsare isolated from one another.
 54. The cooling apparatus of claim 53,wherein the first piping system and second piping system are arranged inparallel in a helical arrangement.
 55. The cooling apparatus of claim53, wherein the interfacing component comprises the conduction surfacethat is configured for attachment to the electronic device, such thatthe thermal interface is provided by conduction.
 56. The coolingapparatus of claim 53, wherein the interfacing component comprises ahousing that defines at least part of a volume in which a third coolantand at least part of the electronic device can be located, such that thethird coolant can provide the thermal interface by convection to theconduction surface that forms part of the heat exchanger arrangement andwherein the third coolant is isolated from the first and secondcoolants.
 57. A cooling system comprising the cooling apparatus of claim34 and further comprising: a piping system, configured to carry a firstouter coolant for acting as the first heat sink of the heat exchangerarrangement and to carry a second outer coolant for acting as the secondheat sink of the heat exchanger arrangement; and wherein the first andsecond outer coolants are isolated from one another.
 58. The coolingsystem of claim 57, further comprising: a second heat exchangerarrangement, configured to receive the first outer coolant and thesecond outer coolant and to transfer heat from the first and secondouter coolants to a common output heat sink.
 59. A method of operating acooling system, comprising: coupling an electronic device to aninterfacing component that defines a thermal interface through whichheat generated by the electronic device can be transferred away from it;operating the electronic device to generate heat that is transferred tothe thermal interface; receiving heat generated by the electronic deviceat a heat exchanger arrangement via the thermal interface; andtransferring heat received at the heat exchanger arrangement to a firstheat sink and a second heat sink; wherein the first and second heatsinks are isolated from one another.
 60. The method of claim 59, furthercomprising: in the event that a part of the heat exchanger arrangementis unable to transfer heat to the first heat sink, transferring at leastsome of the heat that would have been transferred to the first heat sinkto the second heat sink.
 61. The method of claim 59, wherein theinterfacing component comprises a housing that defines at least part ofa volume, the step of coupling comprising locating at least part of theelectronic component in the volume together with a coolant, such thatthe coolant can provide the thermal interface by convection and whereinthe step of receiving heat comprises: receiving a first portion of thecoolant at a first heat exchanger located in a first part of the volume,so as to transfer heat from the first portion of the coolant to thefirst heat sink; and receiving a second, separate portion of the coolantat a second heat exchanger arranged in parallel with the first heatexchanger and located in a second part of the volume, so as to transferheat from the second portion of the coolant to the second heat sink. 62.The method of claim 61: wherein the step of receiving the first portionof the coolant at the first heat exchanger comprises transferring heatreceived from the first portion of the coolant to a first outer coolant;and wherein the step of receiving the second portion of the coolant atthe second heat exchanger comprises transferring heat received from thesecond portion of the coolant to a second outer coolant, the first andsecond outer coolants being isolated from one another.
 63. The method ofclaim 62, wherein the first and second outer coolants are liquids. 64.The method of claim 62, further comprising: receiving the first andsecond outer coolants at a heat exchanging arrangement; and transferringheat from the first and second outer coolants to a common output heatsink using the heat exchanging arrangement.
 65. A method ofmanufacturing a cooling system, comprising: coupling an electronicdevice to an interfacing component that defines a thermal interfacethrough which heat generated by the electronic device can be transferredaway from it; mounting a heat exchanger arrangement on the interfacingcomponent in order to transfer heat from the electronic device to theheat exchanger arrangement via the thermal interface; and configuringthe heat exchanger arrangement so as to transfer heat received from theelectronic device to a first heat sink and a second heat sink; andwherein the first and second heat sinks are isolated from one another.66. The method of claim 65, wherein the step of coupling compriseshousing the electronic device in a volume defined by a sealable module,such that a liquid coolant may be added to the volume for removing heatgenerated by the electronic device and wherein the step of mounting theheat exchanger arrangement comprises: mounting a first heat exchanger ina first part of the volume so that it can receive a first portion of theliquid coolant and transfer heat from the first portion of the liquidcoolant to the first heat sink; and mounting a second heat exchanger inparallel with the first heat exchanger in a second part of the volume sothat it can receive a second, separate portion of the liquid coolant andtransfer heat from the second portion of the liquid coolant to a secondheat sink.
 67. The method of claim 66, further comprising: filling thevolume with the liquid coolant.